[brianwski]

We're considering a PXE boot solution (among other solutions) just to keep all 200 pods (and growing!) updated to the latest Debian. But we also use the boot drive for a few other things like error logging and such. But the idea of eliminating the boot drive entirely is not a bad idea, we could drop the logs in a folder at the top of the data drives. We already (selectively) mirror various excerpts of the logs to other machines in case the whole pod disappears from the network so we have some history and understanding of what was going on when it went missing.

[eropple]

Perhaps streaming logs would help you out? My employer dumps a preposterous amount of log data constantly via streaming log systems (we use log4j, but there are solutions for syslogs, etc.). Aside from a few early hiccups, it works pretty well.