Hacker News new | ask | show | jobs
by KaiserPro 866 days ago
VFX engineer, I have suffered through:

_early_ lustre (its much better now)

GPFS

Gluster (fuck that)

clustered XFS (double fuck that)

Isilon

Nowadays, a single 2u server can realistically support 2x 100gig nics at full bore. So the biggest barrier is density. You can probably get 1pb in a rack now, and linking a bunch of jbods(well NVMEs) is probably easily to do now.
1 comments
nh2 866 days ago
"1PB in a rack"? You can apparently already buy 2.5PB in a single 4U server:

https://www.techradar.com/pro/seagate-has-launched-a-massive...

link
KaiserPro 866 days ago
sorry I should have added a caveat of 1pb _at decent performance_

That seagate array will be fine for streaming (so long as you spread the data properly) as soon as you start mixing read/write loads on that, it'll start to chug. You can expect 70-150iops out of each drive, and thats a 60 drive array (from guess, you can get 72 drives in a 4u, but they are less maintainable, well used to be, thing might have improved recently)

When I was using luster with ultra scisi (yes, that long ago) we had good 10-20 racks to get to 100tb, that could sustain 1gigabyte a second.

link
nh2 866 days ago
Agreed, it depends on the use case. For some "more storage" is all that matters, for others you don't want to be bottlenecked on getting it into / out of the machine or through processing.
link