[sudhirj]

I know the Cloudflare team hangs out here, so thanks, and great job! This was absolutely necessary for my line of work. Couple of quick questions/confirmations:

* R2 will support the same object sizes as S3? We have 500GB+ objects and could go to a 1TB per object. * R2 will support HTTP Range GETs, right?

Egress bandwidth for objects on S3 is the biggest line item on the AWS bill for a company I work for, by an order of magnitude, and this will just wipe it off for the most part.

[greg-m]

Hey, I'm the PM for R2.

Yes to range requests. Current object limit is smaller than that, but we don't have a fundamental restriction there. Shoot me an email to gmckeon [at] cloudflare.

[mnw21cam]

I'd say definitely increase that limit. I have a 4TB file hanging around over here, I'm sure others have bigger ones.

[greg-m]

we will :) we're working to reach 5 GB per upload, with multi-part support.

[danielrhodes]

I personally think uploading massive files is not usually desirable. Better would be an easy way to chunk it and upload and have the server put the file back together, which would increase reliability.

[loxias]

S3 protocol supports multipart upload, which is what you're describing. The client chunks it, the server reassembles.

What I'd like to see is PATCH with multipart/byteranges support. =D

[zackbloom]

Native tus support would be pretty amazing.

[anderspitman]

Tus is over-complicated IMO. Why require the server to store state that's already available on the file system? Just support PATCH requests with an offset parameter. If an upload fails partway through, HEAD the file to see how much got copied to disk and resume from there.

[busymom0]

Really excited for R2 Storage. I am wondering if R2 can solve a limitation I was running into with BackBlaze.

I had tried using BackBlaze 8 months ago as a much cheaper (especially with B2 and CF's Free Data Transfer partnership) replacement for Amazon S3 and was running into a limitation on B2.

I had a scenario where my users can upload images from the browser to BackBlaze. I wanted the ability to control the file name of the uploaded file. I don't want the user to be able to modify the network request to upload the file with a different file name. Nor do I want the users to be able to upload files with names which would overwrite existing files.

B2 didn't let me upload files with a specific filename in the pre-signed URL.

For example there's this API:

curl https://apiXXX.backblazeb2.com/b2api/v2/b2_get_upload_url -H 'Authorization: XXXXXXXXXXXXXXXXXXXXXXXXX=' -d '{"bucketId": "XXXXXXXXXXXXXXX"}'

which gave me an upload URL where I can upload the file to.

And then there's this:

https://www.backblaze.com/b2/docs/b2_upload_file.html

which lets users upload files to that URL.

But this allowed my users to upload file with any name they want. It would also allow them to overwrite existing files (from other users).

My question is more from a security point of view so preventing one user from overwriting another user's content is crucial. For example, lets say you right click on an image from someone else on facebook and get the actual image's file name. Now you try to upload an image on facebook and you edit the network request in the browser's inspector tool to the image file name which you got for another user. Facebook obviously prevents this in their own way using pre-signed urls which include the filename in the signature. However on BackBlaze if I try this, the "pod" url which is received doesn't include any file name signature. The pod URL is just where the image gets stored on your end. A user can easily edit the network request and modify the "X-Bz-File-Name" Header to another user's filename. This would be a major security vulnerability if I went with BackBlaze. As a workaround, right now it seems like users would first have to upload files to my own server, then my server would have to upload them to BackBlaze to avoid this issue. This sounded like hassle.

Amazon S3 solves this problem using createPresignedPost which includes a signature of the filename in the URL. I contacted BackBlaze's support and got a response their S3 api doesn't support createPresignedPost:

https://www.reddit.com/r/backblaze/comments/kzszym/is_backbl...

You can read B2's staff's response to my question:

https://www.reddit.com/r/backblaze/comments/l0c9s7/is_there_...

Is there a way to prevent this on R2? Something where the link provided by b2_get_upload_url (whatever R2's equivalent will be) only works for specific a file name?

[nilayp]

Nilay from Backblaze here. Amongst other things, the solution engineering team reports up to me... and when I saw this comment yesterday, I had them dig into why the B2 S3 API doesn't did not work for you.

It turns out that Backblaze B2 absolutely does support uploading objects via S3 style presigned URLs. However, there are a couple of caveats:

1. B2 doesn't currently support PostObject operations. Therefore, you must use PutObject to upload to a pre-signed URL. Many of the AWS SDKs default to PostObject.

2. B2 only supports AWS v4 authentication and not v2, which AWS has deprecated. However, some AWS SDKs default to v2.

I've put together a very simple python code sample that successfully works for me. https://gist.github.com/nilayp/2c2a04f033d8992ce4b8f591ab449...

Would this solve your issue?

[busymom0]

I appreciate you responding here.

I am currently away for this week, so I can't try it this week. Can I reach out to you next week with my findings? Is there an email address I can reach you at?

Just wanted to mention that I had raised my issues on the backblaze subreddit back then:

https://www.reddit.com/r/backblaze/comments/m9tioi/backblaze...

https://www.reddit.com/r/backblaze/comments/kzszym/is_backbl...

https://www.reddit.com/r/backblaze/comments/l0c9s7/is_there_...

[nilayp]

Absolutely! I’d love to know whether this worked for you.

Drop me an email using my GitHub username at backblaze.com

We will correct the record on Reddit after hearing that this was successful for you.

[SmellTheGlove]

I have this exact same use case for an app I’m building and would love an answer as well. I built on S3 as a result. When a product says “full S3 api compatibility” this becomes my question.

Going further I don’t want to become someone else’s back door file repo for illegal shit. So presigned upload urls with an enforced file name and configurability over the size limit and expiration of the presignedpost (both in terms of time and number of files) is pretty important to me. S3 does a good job here.

[busymom0]

Yes, file size limit is another thing I required in my setup which B2 didn't have.

[nhoughto]

I've always found pre-signed URLs fragile and a pain to work with, when we solved this problem we just put cloudflare workers in front of b2, so the worker can sign requests to b2 itself, so the interface to the user can be simpler. Could just be a straight POST with bytes, then the worker will turn that into a PutObject call to the s3 api, it works pretty damn well.

If you are reading files back out again too you can use cloudflare caching to speed things up, its a good combo.