[sillysaurusx]

Personally I find Backblaze B2 more compelling.

... Ok, after writing that, I realized that I should probably look up R2's offering first. And then my jaw hit the floor at "free egress bandwidth".

Free egress bandwidth? Yes please.

I will instantly convert.

Then I was worried about price per GB. $0.015/GB is incredibly competitive.

Good lord, where do I get early access to this thing? I'd transition all our infra today.

[jacurtis]

According to their press release it says:

> That's why Cloudflare plans to eliminate egress fees, deliver object storage that is at least 10% cheaper than S3, and make infrequent access completely free for customers

A few elements of concern. It says they "plan to", so we don't know if this is near term or some moonshot type of goal. It also states earlier in the press release that the free egress frees are offered via the "Bandwidth Alliance", which removes bandwidth charges between member providers. While it is noteworthy that GCP (Google Cloud Provider) and Microsoft Azure are part of this alliance, AWS is not! So does this mean that egress fees will be charged if an AWS server requests the file?

I also don't know what to think about the statement that they plan to make infrequent access completely free... does that mean that I can throw files onto R2 for archive purposes and not pay anything? Because that is what it sounds like by that statement, but it obviously sounds impractical or too good to be true.

Original Press Release: https://www.cloudflare.com/press-releases/2021/cloudflare-an...

[greg-m]

Hey, R2 PM here - there's no question that the product will have 0 egress charges, regardless of destination.

For archival use-cases, you do still pay us for data storage. We're referring to not charging for operations for infrequent access - we'll likely drop the stored data charge down too, eventually, but the current pricing is complex enough.

[jacurtis]

Ok, thanks for the clarification. After I wrote that statement about Infrequent Access, I was thinking about it more and realized that you probably pay for storage but simply have no access fees. In other words you don't really distinguish between storage tiers. I think that is good. S3 technically has 7 storage tiers, with all permutations of limited availability zone, reduced redundancy, infrequent access, archival storage, etc. While it is understandable that archival storage is unique (it is tape storage), the others just seem arbitrary and unnecessary.

I am an AWS administrator/architect for work so we are always trying to weigh pros and cons. AWS S3 is notoriously overpriced. This is a well-established fact. There are many other providers that offer comparable solutions (or even superior) such as the new R2. But we feel the effect of vendor lock-in because of S3's integration with other AWS services, which is what keeps a lot of people over-paying for S3. I think the auto-migration feature is potentially one of the best arguments for switching to R2.

R2 is undeniably a better value than S3. S3 requires me to select a region and optionally even limit an availability zone (if I need to keep costs low). CDN/edge locations are all extra cost via AWS Cloudfront. And the reality is many people are already using Cloudflare as CDN in front of S3 storage. So R2 just becomes a no-brainer at that point. I think it will be a successful launch. I am excited to try it.

[rsync]

"I am an AWS administrator/architect for work so we are always trying to weigh pros and cons."

Very informal survey ... but I wonder if you are familiar with the 'rclone' tool:

https://rclone.org/

... just curious ...

[sillysaurusx]

Thanks! R2 coming out with $0 egress is somewhere between unbelievable and literal-miracle.

I suppose my only skepticism is "but how fast can I egress?" -- if the bandwidth is 100x slower than GCP, it might dampen my enthusiasm a little bit. But honestly I'd still take a 100x slowdown if it means I can do long term archival without paying $200 just to download the data, soooo....

Anyway, cheers, and thanks for doing impactful work!

[mst]

> R2 coming out with $0 egress is somewhere between unbelievable and literal-miracle.

Not necessarily.

A nearlyfreespeech noted when they changed to a charge-by-incoming-bandwidth model, links are still often bought symmetrically and for them the thing that eats the most bandwidth is incoming DDoS attacks, not legitimate traffic.

It seems to me that Cloudflare are likely to be in this situation except even more so.

[sillysaurusx]

(@CameronNemo: If you want to chat about it, you'll have to DM me on twitter. As you see, comments here about it are instakilled.)

[Matheus28]

I honestly can't see how it'd be profitable if someone were to host several TB worth of files that are very frequently downloaded. My fear with anything "free" is that once you actually use it A LOT, it will be pulled from under you. I'm a lot more comfortable with $0.001/GB than $0/GB.

[prirun]

At high request volumes there would be a per-operation charges, so not exactly free.

Backblaze (the consumer thing, not B2) has always had unlimited backups, and still does. One of their execs (Brian) posted a graph showing a backup space utilization histogram for all customers in 2018. The first few data points are:

30% use <100GB 15% use <200GB 9% use <300GB ...

So ~55% of their customers are using <300GB and paying $6/mo. On B2, their cloud storage product, $6 will buy 1.2TB of storage. Way more than half of their unlimited customers are paying 4x more for unlimited storage than they would if they paid for metered storage.

Interestingly, on that same histogram, are the last 2 customers: one using 293TB and one using 430TB. While IMO they are misusing the service, they are also an extreme minority. If that last guy was paying for that space with B2, it would cost $2150/month. Backblaze has said they just don't care. Now if it were half their users doing it, of course they would.

I can tell you one thing: R2 with free bandwidth is going to be a gift for the porn industry!

[mst]

My guess is that carrier grade links are generally symmetrical and cloudflare's provisioning requirements are driven by incoming DDoS traffic, which makes the economics quite different than they would be otherwise.

[politician]

Hey Greg, I was just talking to Sales about R2 yesterday, and they said that it was not live yet. Can you clarify availability?

[judge2020]

> R2 is currently under development... We’ll be announcing an open beta where any user will be able to sign up for the service soon.

https://blog.cloudflare.com/introducing-r2-object-storage/#:...

In my opinion, expect 3-6 months for the beta, then maybe 2 years for GA. Storage at scale isn't easy, especially if they want to be comparable to aws in speed, SLA, data protection (ie. duplicated to 3+ physical data silos), etc.

[GordonS]

Sorry to side track things, but I have a somewhat related Cloudflare question!

I'm using Azure, rather than AWS, and I hope R2 is available for Azure eventually too.

In the mean time, I came across a Cloudflare blog post from earlier this year that said you can use Azure's preferred routing feature to point to Cloudflare, which will result in "substantially cheaper" egress bandwidth fees. Sounded good, but when I looked at Azure's bandwidth pricing it looks like egress routed through Cloudflare is barely any cheaper, only 9% or so :(

The question: am I missing something, or is that paltry reduction really all Azure is doing?

[stavros]

This is rather offtopic, but what's with your documentation "code" fields being uncopiable? Those are the exact fields I need to copy!

E.g. https://developers.cloudflare.com/cloudflare-one/connections...

[selcuka]

Wow, that's really weird. They seem to have specifically disabled it. You can use the userscript I just made for myself [1] as a workaround for now.

https://gist.github.com/selcuk/90ce1ce8b8d46c869efd6da24cde1...

[stavros]

It is really weird, yeah, expecially since the examples seem to be meant to be copied. Thanks for the script!

[7ewis]

Not sure if it's been fixed, but works fine for me using Brave on Mac.

[tomjen3]

Will you be going after "abusive" hosts? Like if I wanted to use your services to deliver a successful podcast or a viral video I would be paying you pennies a month for storage, while taking up boatloads of bandwidth.

I am not planing on doing either, but I am just curious what you would do about it?

[sodality2]

If the requests /sec are over the free limit, you'd be charged.

[nemothekid]

>While it is noteworthy that GCP (Google Cloud Provider) and Microsoft Azure are part of this alliance, AWS is not! So does this mean that egress fees will be charged if an AWS server requests the file?

The bandwidth alliance means when Cloudflare requests a file from GCP, GCP won't charge you egress. Cloudflare will then deliver your file to your customer for free.

[bbu]

small correction: gcp will still charge you but at a reduced rate.

[GordonS]

Do you know how much the discount is? I was looking into this regarding Azure just yesterday, but it looks like they only discount the egress bandwidth by a pitiful 9% or so.

[spullara]

I think they mean the cost for access would be completely free (if not accessed much) but not the storage as those are separate fees.

[sillysaurusx]

Responding to a deleted but interesting comment. Normally I wouldn't do this, but it's harmless enough:

> Backblaze B2 transfer to Cloudflare is free egress due to bandwidth alliance and Cloudflare CDN is free egress. So you already kinda have that.

Hmm... What does it mean to transfer to Cloudflare? That's interesting.

I want free egress to my Hetzner servers. (4x16TB for $79EU/mo is unbeatable, primarily because Hetzner also has unlimited free egress bandwidth.)

But if Cloudflare offers servers, I should look into that. Do they? Even if they do, what's their egress pricing?

Thanks for the tip!

(I've been wondering about B2's mysterious "computing partners" -- their computing partners get free egress, so it seems entirely plausible that Cloudflare might be one such computing partner. I just didn't realize that Cloudflare might do computing at all -- in my head, they were a proxy, not a server farm.)

[DeathArrow]

They offer a serverless platform.

If you want free egress, you can check Wasabi: https://wasabi.com/cloud-storage-pricing/#three-info

[Polycryptus]

I've been using Wasabi for a project where cost is more important than anything (i.e. a side project with reasonable scale that can't accept income) and on that front it is great. (S3 costs have been atrociously high to us sometimes.) For CDN-type resources we do a Wasabi bucket with Cloudflare with caching set very high. Reliability has been the only problem; it's not awful by any means, but there are a lot more "hiccups" using it. You get what you pay for, I guess.

That said I'm looking to see if just using R2 is a big improvement. It'll cost more but the reliability and performance might be worth it for us.

[NicoJuicy]

Wasabi charges for your largest file for 3 months.

You only get "free" egress for the size of your storage.

[prirun]

Correction: Wasabi charges 3 months storage for all files. If you upload a file and immediately delete it, you pay for that storage for 3 months.

[sillysaurusx]

Thanks for pointing this out!

[prirun]

Vultr is a Backblaze compute partner. I've used both for years and never had a problem.

Another nice thing about B2 is that they have a 1GB free daily egress allowance, which is handy for backup programs. HashBackup (I'm the author) uses that for downloading and repacking backup data to optimize storage efficiency.

[mattjaynes]

As mentioned in another comment, don't forget to consider B2's weekly 2 hour maintenance windows during the US work day: https://www.backblaze.com/scheduled-maintenance.html

[nvahalik]

This was the reason we ended up moving away from B2. It may be durable but their storage isn't a drop-in replacement for S3 or similar products. They are really an S3-compatible target for backup softwares.

[SiVal]

This would be fine for an automated system that could schedule requests or could fail, wait 30min, and try again. But it would be unusable by a human working interactively. You would constantly get complaints and monitor warnings that your "system is down".

I don't mind services that cut price dramatically in exchange for only supporting some (not all) use cases. That can be a very attractive offer, but does Backblaze explicitly position B2 for non-interactive use cases?

[atYevP]

Yev from Backblaze here -> that maintenance window doesn't shut down the B2 Service each Thursday for two hours. We have lots of companies like Kanopy and Cloudspot that use us as an origin store without issue!

[forty]

We will see in practice how it is. As the article suggested, there are often caveats to "unlimited" and honestly I don't even think they have written unlimited anywhere :)

[aclelland]

I think that when CF say unlimited bandwidth they really mean it. I manage a domain on the business level plan for a domain and I pushed over 1PB through it in January. Not a single complaint from CF and no sales calls pushing enterprise tier.

They haven't clarified their file operations costs yet though. That could get pricy but will more than likely be cancelled out by the egress savings for most use cases.

[Aea]

Not saying your experience isn't true, but I've heard horror stories of accounts being disabled for using too much "non-HTML" bandwidth, even on business level ($200/month) accounts (at the single digit TB level). The limits seem to be arbitrary and ill defined.

CF may be great technically, but I personally wouldn't use them without an enterprise agreement in place. Bandwidth should be cheap, but cheap does not equal free.

Unless I had an enterprise agreement in place I'd rather work with a vendor that has a well defined usage-based pricing. I have a low appetite for risk, and usage-based pricing aligns incentives properly IMHO.

[aclelland]

Yes, you are right that an enterprise agreement is probably the safest approach and it's definitely something we have looked into since the beginning of the year.

In our case, one of our games DAU went pretty crazy last Christmas which resulted in a huge increase in players (who all need to download hundreds of MB of data). Maybe if it'd continued for many months the situation would be different and that angry email from CF would have eventually arrived.

[breakingcups]

Yeah, that's why I haven't dared put them in front of my B2 buckets, even though they have the Bandwidth Alliance.

[jonny_eh]

> I personally wouldn't use them without an enterprise agreement in place

That would depend on the use case I'd assume.

[corobo]

How long ago was that?

[Aea]

Came across these 3-4 years ago when I was doing research on whether CF would be viable for a previous company.

[fernandotakai]

>As the article suggested, there are often caveats to "unlimited" and honestly I don't even think they have written unlimited anywhere :)

CF already talked about egress pricing here https://blog.cloudflare.com/aws-egregious-egress/

i would say that when they talk unlimited, they actually mean it.

[forty]

Precisely my point is that I haven't seen "unlimited" mentioned either in the R2 announcement nor on your link. So it's not unlikely that it's going to be free egress up to X GB per month (possibly with X high enough that it's still cool)

[Eugr]

Did you have a look at rstor.io? They have free egress and even lower storage costs and also don't seem to charge for IOPS. Not as high profile as Cloudflare though.

[Smerity]

Thanks! I'd never seen them. The only minor catches that I've seen are that they want you to start the account with $50 or more added, it's $6 per TB per month (rounded up to the nearest TB) with a 90 day minimum duration, and you're allowed 1TB egress per day for each TB you have stored.

They were also acquired very recently[2] though no clue how that might impact things.

I'm fine with all those except the pre-loading $50 just to test, though that's as I want to test it personally. If I were using a business account that's not as much an issue.

[1]: https://console.rstor.space/pricing

[2]: https://www.yahoo.com/now/packetfabric-announces-acquisition...

[iptrans]

”you're allowed 1TB egress per day for each TB you have stored.”

Where did you find the reference to 1TB per day?

On the page it says:

“1 TB of data egress for every 1 TB storage capacity used”

I would have assumed this is per month.

[Smerity]

You're right, I misread it and now can't edit my comment. This puts it all in quite a similar position to Wasabi[1] who have a 1:1 storage:transfer ratio per month and a 90 day minimum storage duration as well. In fact, their price per terabyte is almost exactly the same ($5.99 for Wasabi vs $6 for Rstor).

It's a shame as I love aspects of this type of storage service, even with the caveats, but they're not useful if there's no way to pay more for excess transfer. Luckily I think R2 fits that requirement (though paying for more operations vs paying for more transfer).

[1]: https://wasabi.com/paygo-pricing-faq/

[Eugr]

That's interesting... When talking to their reps I mentioned Wasabi and asked if they have similar limitations and they said "no". I wonder if enterprise customers are treated differently?

[Smerity]

From the pricing page I linked to[1]:

> There are no charges for outbound data transfer when using a dedicated network connection to the RSTOR network and direct peering is established. These connections are charged separately. To find out more please contact us.

I would imagine for enterprise customers you might either be using direct peering / similar or they're able to charge based upon ballpark bandwidth usage. Otherwise I'm unsure as it does seem quite in line with Wasabi otherwise.

[1]: https://console.rstor.space/pricing

[DeathArrow]

Also Wasabi is worth checking. Free egress cheap prices. https://wasabi.com/cloud-storage-pricing/#three-info

[breakingcups]

If I recall correctly, that's only good up until the total storage. So if you store, say, 100GB of files total, you only have free egress up until 100GB/month

[encryptluks2]

Exactly... also, they have like a 3 month minimum storage time on a lot of their plans.

[sillysaurusx]

I haven't! Thank you for the tip!

[dcolkitt]

Free egress is huge for my data pipeline workloads. I hate getting silo-ed into a single cloud provider, because my data lives there and it's too expensive to run the compute jobs on another cloud due to the cost of moving the data.

For example GCP has a much better Kubernetes offering than AWS, but everything's native to S3. So you get stuck using crappier products. If R2 offered free egress, I'd move the entire data lake there just to sidestep this problem.

[true_religion]

OVH storage is $0.0112/GB and egress is just $0.01/GB. Having egressed priced out is handy because then you can just use it directly.

With Cloudflare you'd have to pay for the CDN separately for any real traffic amount.

[sydney6]

Backblaze, IIRC, doesn't distribute data across multiple regions.

[kaixi]

B2 to Cloudflare egress is free, they are both part of Bandwidth Alliance: https://www.backblaze.com/b2/solutions/content-delivery.html

[austinpena]

Answered here: https://news.ycombinator.com/item?id=28778286

[hitekker]

B2 is also a lot slower egress than S3, by maybe an order of magnitude.

[prirun]

I'm author of HashBackup and regularly run performance comparisons between object storage services. While I agree than S3 performance is generally better than B2, I personally have not seen a 10x difference in egress performance.

Here's a test I recently ran comparing S3, B2, and Storj. The test uploads a 700MB backup, downloads it and rebuilds a database, then removes it. The test was run on a small 1 CPU, 512M Vultr VPS.

  S3 upload: 20.6s
  S3 download: 26s
  S3 remove: 1.2s

  B2 upload: 28.2s
  B2 download: 28.7s
  B2 remove: 4.0s

  Storj upload: 34.7s
  Storj download: 75.3s
  Storj remove: 20.6s

B2 does have higher latency than S3, so for short operations like removing objects, S3 has higher performance. If your egress load is for a lot of small objects, I could see B2 being a lot slower. This shows up in my tests where removing 14 backup files is more than 3x slower on B2 than on S3. But even for this, I've never seen 10x slower.

A footnote: the Storj tests are with a new driver so those numbers are preliminary results.

[irrational]

Egress means files being requested, right? Is there a charge for sending files to the server (ingress)? Does this mean the only charge is the storage space?

[sillysaurusx]

This was one of the most surprising things, to me, before I learned about any of this stuff.

inbound traffic (ingress) is free, almost universally.

outbound traffic (egress) is $insane, almost universally.

It's where most of the cloud providers make most of their money, as far as I can tell.

I'm a fan of murdering egress fees, so therefore I am a fan of whatever Cloudflare R2 turns out to be. As long as they get rid of egress, I'll cheerlead them for life.

(To answer your question more directly: "yes, I think so. At least in my experience.")

[anyfactor]

I googled what egress meant.

> when that data is retrieved from the cloud, providers will then charge large fees; this is what’s known as a data egress.

There are many cloud providers seem to be not charging for egress.

[the_duke]

No, most cloud providers do not charge for ingress, but charge heavily for egress.

[anyfactor]

I didn't mention or didn't know what ingress is.

The full excerpt [0]

> Most leading cloud providers allow their customers to input data into the cloud for free. However, when that data is retrieved from the cloud, these providers will then charge large fees; this is what’s known as a data egress.

Here is what I found out about ingress [1]

> Egress in the world of networking implies traffic that exits an entity or a network boundary, while Ingress is traffic that enters the boundary of a network.

[0] https://wasabi.com/help/glossary-of-terms/egress-charges-def...

[1] https://aviatrix.com/learn-center/cloud-security/egress-and-...

[sillysaurusx]

no worries, it was all quite confusing to me too. I never had to worry about any of this stuff till accidentally racking up a $600 charge in one day by using a TPU pod to train on data in the wrong region.

Here's the rule of thumb: https://news.ycombinator.com/item?id=28775836

> inbound traffic (ingress) is free, almost universally.

> outbound traffic (egress) is $insane, almost universally.

so think of it like, teleport yourself to S3's servers. Any data that comes in, you charge $0. Any data that goes out, you charge $massive.

This seems to be true for almost every provider I've found. Hetzner is one of the rare exceptions. If you need a server, get a Hetzner dedicated box, because it's unlimited traffic (both ingress and egress). It powers https://battle.shawwn.com/ (big dump of files).

I'd love to store things in S3 or GCE, but it's a non-starter, because transferring between GCE to Hetzner would cost $0.12/GB downloaded. 12 cents per GB! It doesn't sound like a lot till you do the math on 22TB.

Hence, R2 is incredibly appealing. I'd love $0.015 per GB storage cost + free egress, because it means I can download as much as I want to my hetzner server. Meaning, I don't need to worry about my hetzner drives failing.