Reading the API, it seems that I need to precalculate a SHA-1 before uploading? This makes it impossible to stream data to b2 from another source, I'll need to store it first then send to b2.
Right now your only option would be to "buffer" packets of say 1 MByte in RAM, calculate the SHA-1, then store them as separate files in Backblaze B2.
We do plan to add file offset access and larger file support very soon, so you would be able to append a 1 MByte chunk to an existing file in Backblaze with a SHA-1 of only the 1 MByte chunk. That should allow you to stream?
All great feedback, by the way. We really want to hear about these shortcomings in our API right away.
Having to know the SHA1 in advance would be a show stopper for rclone ( http://rclone.org ) as it uses a streaming model internally (it can stream between cloud providers).
Being able to append to a file in 1 MByte chunks (or larger) would be perfect - that is exactly the way Amazon S3 multipart uploads and google drive multipart uploads work.
I'm planning on storing files as encrypted X megabyte chunks (where X is TBD) - would calculating per-chunk and then uploading solve it? There's metadata support which could store original filenames etc.
> We do plan to add file offset access and larger file support very soon, so you would be able to append a 1 MByte chunk to an existing file in Backblaze with a SHA-1 of only the 1 MByte chunk. That should allow you to stream?
We do plan to add file offset access and larger file support very soon, so you would be able to append a 1 MByte chunk to an existing file in Backblaze with a SHA-1 of only the 1 MByte chunk. That should allow you to stream?
All great feedback, by the way. We really want to hear about these shortcomings in our API right away.