[bisby]

I once had an employer that required us to use passworded SSH, and disallowed SSH keys, because they couldn't enforce that the SSH keys were passphrase protected, so just turned that option off.

They said it was a PCI requirement, or something.

[yardstick]

PCI requires multi-factor auth these days, so you’ll likely find now the ssh password will be your password plus a OTP at the end.

[mazone]

PCI DSS from 4.0 actually have something called customized approach for everything. If you can prove and the QSA agrees that you fullfill the goal of a requirement, you can be quite flexible. Example i am doing things like not using passwords at all and only passkeys, or only ssh keys protected by hardware security key etc. Together with agents trying to verify the devices connected are company owned and hardened in different ways. Your milage might vary depending on how good your auditor is but PCI DSS standard do have quite a bit of flexibility in it.

[yardstick]

Presumably at some point in your environment you are doing MFA? Just not at every step?

Ie If someone broke into your office, opened computer, inserted the hardware security key, would they get in? Or is there something else non-physical going on? Like the initial login is password + security key, and you can demonstrate the ssh keys never leave the secured PCs etc.

[mazone]

It is not about MFA or not but to demonstrate the process is secure for the purpose.

It can be complicated but a example. TOTP that is very common used with passwords is regarded as MFA (tho most of the time software based on phone) but have many problems regardless

- many time replayable - can be intercepted - implementations look different - recovery code reuse problems etc.

On the other hand, using only passkeys dont have those problems but with passkeys, many times you cannot decide on what device a user have registrated the passkeys in a enterprise setting. example they could be apple passkeys, chrome passkeys, windows, hardware key(yubikey) etc and all of them behave different when it comes how they ex can be copied/ synced between users devices. So from where they can be used.

So for any authentication flow, you need to look at the full picture. What is the process when credentials are lost? How do user onboard etc.

Is a good entry point to say. We should use MFA or similar but the details matter.

[yardstick]

It’s always interesting to hear other people’s approaches- Thanks for the insights!

[notpushkin]

Isn’t there a way to ask for OTP after initiating the SSH session?

[odo1242]

One issue with this is it makes it hard for scripts to log in (this may be what you want)

[mrspuratic]

Yes, via PAM, and this works fine with OpenSSH. But the couple of OTP implementations I've used are the same, you can either provide password and PIN or passwordPIN. In the end they get concatenated, passed to the next layer, and taken apart for checks. This lets it work with brain-dead http basic auth too, if you're unlucky enough to have to use that...

[notpushkin]

Yeah, I’ve seen it in a couple places, too. (Once I’ve had to write my own wrapper for openfortivpn that did exactly that!)

[jerf]

This is not as illegitimate as it may sound to you. You may not hear about "getting someone's SSH keys" very often, because we only hear about "vulnerabilities" on places like HN and this isn't a "vulnerability" in any software.

But getting someone's SSH keys and then running off and doing other things is a very normal part of any focused attack in which attackers use some foothold to start pivoting into your systems. It's one of the first things an attacker will check for, precisely because it's high likelihood they'll find one and high reward if they do. It's an extremely serious threat that you don't hear about very often, just like you may not hear about "the sudoers file left something open with passwordless access it shouldn't have and the attackers lifted themselves up to root from there" even though it's a major part of many actual incursions. I'm aware of multiple cases in which someone's passwordless SSH key played a part of the process.

So that really is a legitimate problem and turning them off is not security theater but can have a real impact on your security posture. The problem is solved nowadays with adding other auth to the process like proving possession of a physical token as part of the login process.

[seadan83]

> But getting someone's SSH keys and then running off and doing other things is a very normal part of any focused attack in which attackers use some foothold to start pivoting into your systems.

Though, if someone gets that far, couldn't they also install a key logger on the users system? At that point - whether it's just password or a password enabled SSH key, anything the user does is all compromised regardless.

[jerf]

"Though, if someone gets that far, couldn't they also install a key logger on the users system?"

There are a ton of situations where that is not the case. They may be using directory traversal from something else to read a key without even necessarily being on the system. They may be on the system at 1am local time, and want to get in and get their job done before the user is even there. They may be on a server somewhere where someone left a key they shouldn't have. The attacker may have gotten enough of a secret to compromise some other secret store where the key is being held. They're probably on a system with user-level access only and that may not be enough to "just" install a keylogger, depending on how the system is set up and how the user accesses it. These are examples and not even remotely a full enumeration of all the possibilities. I won't tell you which ones, but some of these are things I've personally seen attackers take advantage of, so they're not just theory, either.

When you're under personal attack, not just getting popped by some vuln scanner scanning over the entire Internet, the situation becomes very different than a lot of people here on HN are used to. Ever been locked out of a system accidentally, then thought for a moment and strung together three other things to reach back into the system you were locked out of, like "oh yeah, I can push a software update to this automatic deployment system, which will run a bash script that checks the IP address and if it is this system restarts the ssh server, and so after 10 minutes we should be in"? Imagine someone who does that every day, all the time, as their full time job, and then imagine they're on a team of other people who also do it every day as their full time job, then imagine they've gotten a foothold into your system. Which, by the way, they immediately used to put a command-and-control client on your system, loaded with all kinds of exploits, and the ability to push arbitrary code to any number of systems at a time and all the tooling to use that as if they've been developing it for 20 years, which they have. What's the transitive closure of what they could work out how to access? The answer would probably surprise you.

[seadan83]

I appreciate the additional insights, but the premise I'm pushing back on is whenever a SSH key is read, then the user account is by necessity compromised in order to do so. Given that level of a breach, there are myriad ways for an attacker to escalate privilege and exploit their access without worrying about a password on the SSH key. Namely, at that point, cracking the password on the SSH key is a tractable problem.

> They may be using directory traversal from something else to read a key without even necessarily being on the system.

At least on linux - to read the directory containing a SSH key requires the ability to also write to that directory, as the user. Therefore you can also write to '.bashrc' and all sorts of other places. I suspect Windows might have a larger attack surface, but nonetheless, a directory traversal that is able to read and write is also able to install a keylogger.

> They may be on a server somewhere where someone left a key they shouldn't have

Private key should never be transmitted over a network boundary. SSH key passwords can be bruteforced as well. Having a password on the SSH key, when the SSH key is somewhere it really should never have been, is closing the barn door after the horses have left.

> The attacker may have gotten enough of a secret to compromise some other secret store where the key is being held.

Again, getting access to the secret is enough to also have write access and be able to install a key logger. A password on the SSH key still does not help.

> They're probably on a system with user-level access only and that may not be enough to "just" install a keylogger

If a person has enough access to read a SSH key, they can also install a key logger for at least that user account. They are equivalent levels of compromise, a user account having its SSH key read is already compromised.

edit: addendum: There are certainly attacks that can only read the contents of a system, with root that can read the full system. It's just odd to think about, since at that rate the SSH keys being on a prod system is already such a big no-no. SSH keys really need to live exactly just on the personal devices of the people who own those keys - EG: it should never be the case that say a SQL injection attack that gains root level read permission over everything on a filesystem can then ever read SSH keys - cause those keys should never be on the remote system to begin with. Putting a password on private keys that are then copied to servers _is_ security theater; the keys ought to never be copied to a remote server to begin with.

[jerf]

I can tell you've not been involved in defending against an active attack. You, as the defender, do not get to play the game of "well, if I squint and read it that way, that attack wouldn't work". The attackers get to play "well, hey, if it turns out I do this and that and push it through the other thing, I get access". They are the ones who get to flow through any crack they can find. They are the ones who get to do logic chopping like you're trying to do. You don't get to argue "Well gosh, that team shouldn't have left that one permission open on that one system, that's not a best practice, if they'd followed best practices 100% of the time the attackers couldn't have gotten in...". Your job is to pick up the pieces.

[seadan83]

> I can tell you've not been involved in defending against an active attack.

Um... I'm really happy you can mind read dude.

The whole premise here is whether passwords on SSH keys actually help.

SSH keys tend to live in two places: (1) a developers laptop, (2) some sort of CI/CD machine.

The passworded SSH key helps in case (1) only when a person walks away from the laptop and leaves it unlocked. An attacker can't simply then open up a SSH terminal and then SSH to whatever they see in the history. Or, it helps in case a person has a laptop that never locks and the org is simply trying to buy enough time to rotate the persons public keys before the SSH key password is cracked. The SSH key password can buy time, yes - but it does not change the actual security posture.

In case (2), all sorts of considerations need to be made. Though, any password would need to be encoded in a way that is just as accessible as the secret key itself. In case (2), the password really does nothing.

So, yeah, passwording a SSH key does not really help very much. If the keys are left around, then it is the fact they are left around that is a problem. The solution is not to create a scanner that all SSH keys have a password on them, but instead be sure that no SSH keys are installed on systems where they do not belong.

> They are the ones who get to flow through any crack they can find.

I would agree, that is why defense in depth is a good principle just as zero trust security.

So.. passworded SSH keys are kinda really security theater. Please give an attack vector on a persons laptop where the password SSH key is going to stop an attack that is otherwise unachievable. I'll steel man the counter position and mention that physical access is one case (except in that case we are only buying time). AFAIK, that is really it, that is the one place where a password on a SSH key helps.

To hack a persons laptop and read contents in the .ssh directory is a full compromise. To pick up the pieces after that, you need to do things like make sure the compromise can be observed, that privilege escalation is limited. If it comes down to the password on a SSH key being last line of defense - it's game over.

Keep in mind, your burden of proof is to show cases where SSH password actually provides a true increase in real security. The meta conversation of "well, you obviously never have.." is not interesting.

I'll note, if it is the case that SSH passwords are actual security - then, presumably you would feel comfortable stating "yeah, the SSH key was stolen, but nothing needs to be done because the SSH key had a password on it!" It's like the locks on doors, it is for the honest people, the criminals are only slowed down but not stopped by most door locks.