|
|
|
|
|
|
by lamontcg
1252 days ago
|
|
|
The bigger problem is backing things up. The old image level 3 servers at Amazon were just image files layed down in a filesystem (hashed, with a directory heirarchy so that massive numbers of files per directory were not the issue). The problem that it reached was that you couldn't ever take one of them offline and you couldn't stream off of one of the block devices, so you were stuck enumerating through all the files. Those were something like 32kB average filesystem (or possibly slightly smaller). And that was on spinning rust with something like a 4ms seek time between files, and the end result was something like a couple months to go through the whole filesystem. This is why the GoogleFS paper uses chunk sizes of something like 64MB so that data can be efficiently streamed. |
|
|