Hacker News new | ask | show | jobs
by tashian 1179 days ago
It's easy to say "should have used an HSM" (or, in truth, many HSMs), but I can appreciate the technical challenges of acutually doing that at their scale. It would not be a trivial project. There's a ton of operational concerns here, including figuring out how you would go about rotating the key on all those HSMs in an emergency.
2 comments
lxgr 1179 days ago
These would also need to be very distributed and high-throughput HSMs: You'd need to talk to one for every single SSH login! This is in contrast to e.g. having a CA signing key in a HSM, but distributing keys signed with it more widely.

I suppose (Open?)SSH's PKI mode could support a model like that, but as others have noted here, this requires much more manual work on the user's side than comparing a TOFU key hash.

Maybe that model could be extended to allow TOFU for CAs, though? But I think PKI/CA mode is an OpenSSH extension to the SSH protocol as it is, and that would be a further extension to that extension...

link
tashian 1179 days ago
SSH CAs would make the challenge a lot easier. It sounds like they are using RSA keys here for the widest possible compatibility, and while OpenSSH's certificate support is not at all new, it still may be too new for this application.
link
lxgr 1179 days ago
Using SSH certificates would tie every Github user to OpenSSH extensions though. I'm not sure if many git clients use something else, but it's at least worth a consideration.
link
JeremyNT 1178 days ago
There's a lot of daylight between "use a HSM" specifically and "use a system that prevents junior developers from accessing the key and checking it into public repos."

Storing the key in some kind of credential vault that can only be accessed from the hosts that need it at startup would usually be enough to prevent this particular kind of error (unless you're giving root on those boxes to people without enough sense to avoid checking private keys into git, in which case you've probably got worse problems).

link
wlonkly 1178 days ago
I'm far from junior, and that's far enough to know that this kind of error is very much not limited to junior developers.
link