Hacker News new | ask | show | jobs
by crazygringo 1179 days ago
But not for backing up 5M files. Again, the latency per-file will absolutely kill you.

Just for performance, if you insist on backing up the content of 5M files to Google Drive, you really need to write a custom script to tarball and compress whole directories on-the-fly or something.

I mean, 5M files is like multiple sets of map tiles for the entire world. Sometimes people definitely need to work with those, but using cloud backup through the Google Drive API interface is just not the way to go there...

1 comments

This is not for you or Google to decide. And no, I can sync changes without having to tarball. I might only touch one of those 5M files a week actually, depending on my structure.
Well of course it is for Google to decide. They're the ones running the service. ;)

But again, if you ever want to retrieve all 5M files, I hope you're willing to wait a few weeks-to-months, because that's what it's going to take...

Which is why it's a terrible idea not to tarball them first to something more manageable in terms of latency per file operation.

It was not for them to decide up until now. And yes of course users were willing to wait $duration for it until now. If you are suggesting a work around that is okay. It is not okay for google to tell their users to tarball "now" "because".