Hacker News new | ask | show | jobs
by atYevP 2451 days ago
Yev here -> You're saying that when you moved data over without it changing, the data was re-uploaded? It should deduplicate as long as there's been no changes. If you do see that behavior, please ping support so they can investigate -> https://help.backblaze.com/hc/en-us/requests/new
1 comments
sib 2451 days ago
Yes, that's exactly what happened. It required re-uploading more than 300GB of data.

Will reach out to support.

link
brianwski 2451 days ago
Disclaimer: I work at Backblaze.

> It required re-uploading more than 300GB of data.

What should occur is that it must READ all of the files to make sure it has transmitted them already, which can take hours sometimes, but only a tiny, tiny amount of data is actually transmitted to the datacenter. The client basically shows endless streams of files flowing through it and saying "Currently Backup Up: puppy.jpg" but it isn't really transmitting the files, just verifying the contents haven't changed.

One way to realize it is doing this is watch a network monitor of some kind. Another is if it is going "impossibly fast", like you only have a 10 Mbit/sec upload pipe and it appears to be uploading at 100 Mbits/sec.

link
sib 2451 days ago
Yes, makes sense that it has to read them (and presumably calculate a hash and send that hash to your service) to deduplicate; in this case, it took multiple days to complete the process. This is about how long it took to upload the directory in the first case. I did file a ticket after Yev's reply.
link