Yev from Backblaze -> We don't need the boot drive to be that "large", but it's more of a pricing thing. An SSD, even at 256GB would be more expensive than those 5400RPM drives, so we just go with what's cheap :)
Reliability has a cost, too. You can get a 60GB SSD for cheap (Amazon suggests it's about the same price as the slow 500GB HDD), and the SSD will fail at a much lower rate than that HDD.
Yea fair point, might just be easier to order the 500GB ones through our suppliers along w/ our hard drive purchases, but yea, we might go down to SSDs soon. We're constantly iterating on these things.
It doesn't sound like the boot drive is a single point of failure since data is stored Reed-Solomon coded in chunks across many pods. If one data drive fails, the whole pod has to go down for maintenance for it to be replaced. The only difference is that you get to choose when to take a pod down for maintenance to replace a data drive.
That's the situation I was talking about, yes. It's much cheaper to go into the datacenter once a month to replace failed data disks than to have to go in to promptly replace any system disk HDD that fails, in order to not have 60 idle disks.
Not 100% sure if we've given that any thorough testing, but the ops team is looking at most available options before building these out. That said, if they haven't tried it, they'll read this and give it a go :)
Another option -- check out the very low profile USB thumbdrives (the one I've got sticks out about 4mm or so). Or see if you can get motherboards with an internal USB or SD socket. Maybe combine this with PXE boot for initial setup (the pod gets built with a blank internal thumbdrive, and on first boot it installs the Linux kernel and initial ramdisk image on the thumbdrive).
Edit: It looks like the motherboard you use does have a USB header on the motherboard, where you can plug in a boot thumbdrive.
Supermicro also sells a sata plug mounted storage device it would have enough for a kernel to be loaded.
though i think a PXE or similar boot option would be a smarter move. just enough to bootstrap something to run in ram. (alternatively a minimal kernel and dedicate a section of the disks in the chassis to holding the running software (500MB x 45 with decent levels of erasure coding is a fairly minimal slice.) the kernel could then pull down the code after first boot (or updates)