Hacker News new | ask | show | jobs
by tytso 3262 days ago
A number of Linux kernel developers have been working with a subset of the Usenix FAST (File systems and Storage Technologies) community. We hold a Linux FAST workshop after the FAST conference for the past few years.

A few years back, some of the researchers (professors and graduate students) claimed they were interested in more testing and possibly taking some of their work (Betrfs[1], specifically), and productionalizing it. In response, I spent a lot of time with kvm-xfstests[2] and gce-xfstests[3][4] testing infrastructure, cleaning them, making them work in a turn-key fashion, and providing lots of documentation.

[1] http://betrfs.org

[2] https://github.com/tytso/xfstests-bld/blob/master/Documentat...

[3] https://github.com/tytso/xfstests-bld/blob/master/Documentat...

[4] https://thunk.org/gce-xfstests

Not a single researcher has used this code, despite the fact that I made it so easy that even a professor could use it. :-)

The problem is that trying to test and productionalize research code takes time away from the primary output of Academia, which is (a) graduating Ph.D. students, and (b) writing more papers, lest the professors perish. (Especially for those professors who have not yet received tenure.) So while academics might claim that they are interested in testing and trying to get their fruits of the research into production code, the reality is that the Darwinian nature of life in academia very much militates against this aspiration becoming a reality.

It turns out that writing a new file system really isn't that hard. It's taking the file system, testing it, finding all of the edge cases, optimizing it, making it scale to 32+ CPU's, and other such tasks to turn it into a production-ready system which takes a long time. If you take a look at how long it's taken for btrfs to become stable it's a good example of that fact. Sun worked on ZFS for seven years before they started talking about it externally, and then it was probably another 3 years before system administrators started trusting it with their production systems.