|
|
|
|
|
|
by sliken
1252 days ago
|
|
|
Sure, that helps, for looking up a single file. Doesn't however help with ls or du. Even things like what are the 10 biggest files in this directory are painful. I've seen numerous efforts (Microsoft and BeOS spring to mind) to replace the filesystem with a database. Not aware of any big successes though. |
|
|
The old image level 3 servers at Amazon were just image files layed down in a filesystem (hashed, with a directory heirarchy so that massive numbers of files per directory were not the issue). The problem that it reached was that you couldn't ever take one of them offline and you couldn't stream off of one of the block devices, so you were stuck enumerating through all the files. Those were something like 32kB average filesystem (or possibly slightly smaller). And that was on spinning rust with something like a 4ms seek time between files, and the end result was something like a couple months to go through the whole filesystem.
This is why the GoogleFS paper uses chunk sizes of something like 64MB so that data can be efficiently streamed.