[quatrefoil]

If the exploit wasn't baing used, the odds would would be pretty low. They picked the right place to bury it (i.e., effectively outside the codebase, where no auditor ever looks).

That said, if you're not using it, it defeats the purpose. And the more you're using it, the higher the likelihood you will be detected down the line. Compare to Solarwinds.

[londons_explore]

I suspect I could have used this exact attack against 10,000 random SSH servers spread all over the world, and not be detected.

Most people don't log TCP connections, and those that do don't go through their logs looking for odd certificates in ssh connections.

And no common logging at the ssh/pam level would have picked this up.

Your only chance is some sysadmin who has put 'tripwires' on certain syscalls like system(), fork() or mmap() looking for anything unusual.

Even then, they might detect the attack, yet have no chance at actually finding how the malicious code loaded itself.

[amscanne]

There is no ‘system()’ syscall, and fork/exec would be extremely common for opensshd — it’s what it does to spawn new shells which go on to do anything.

I’m not arguing with the point, but this is a great place to hide — very difficult to have meaningful detection rules even for a sophisticated sysadmin.

[ivlad]

This would be execve() that did not go through PAM dance and end up being privileged process.

I _think_ it’ll look very different in ps —-forest output.

[amscanne]

It’s true that there’s a precise set of circumstances that would be different for the RCE (the lack of a PAM dance prior, same process group & session, no allocation of a pseudo-terminal, etc.). My point was merely that I don’t think they are commonly encoded in rule sets or detection systems.

It’s certainly possible, but my guess is sshd is likely to have a lot of open policy. I’m really curious if someone knows different and there are hard detection for those things. (Either way, I bet there will be in the future!)

[ivlad]

I am trying to figure out if auditctl is expressive enough to catch unexpected execve() from sshd: basically anything other than /usr/bin/sshd (for privsep) executed with auid=-1 should be suspicious.

[matrix_overload]

With sufficient data points, you can do A/B and see that all affected systems run a specific version of Linux distro, and eventually track it down to a particular package.

[bastawhiz]

Unless you're the bad actor, you have no way to trigger the exploit, so you can't really do an a/b test. You can only confirm which versions of which distros are vulnerable. And that assumes you have sufficient instrumentation in place to know the exploit has been triggered.

Even then, who actually has a massive fleet of publicly exposed servers all running a mix of distros/versions? You might run a small handful of distros, but I suspect anyone running a fleet large enough to actually collect a substantial amount of data probably also has tools to upgrade the whole fleet (or at least large swaths) in one go. Certainly there are companies where updates are the wild west, but the odds that they're all accessible to and controllable by a single motivated individual who can detect the exploit is essentially zero.

[guenthert]

There are those who run sshd on a non-standard port and log all attempts to connect to the standard port though.

[Hackbraten]

Those connection attempts wouldn't ever reach the daemon though, let alone get to preauth. So how would an exploitation attempt even be distinguishable from, say, a harmless random password guess if neither ever gets to see the daemon?

[lll-o-lll]

> That said, if you're not using it, it defeats the purpose.

Not if this was injected by a state actor. My experience with other examples of state actor interference in critical infrastructure, is that the exploit is not used. It’s there as a capability to be leveraged only in the context of military action.

[sunshine_reggae]

And that leads to the question:

Why do non-friendly state actors (apparently) not detect and eliminate exploits like this one?

Supposedly, they should have the same kind of budgets for code review (or even more, if we combine all budgets of all non-friendly state actors, given the fact that we are talking about open-source code).

[phire]

How to you know they don't?

When a state actor says "We found this exploit", people will get paranoid and wondering if the fix is actually an exploit.

Not saying it happened in this case, but it's really easy for a state actor to hide an extensive audit behind some parallel construction. Just create a cover story pretending to be a random user who randomly noticed ssh logins being slow, and use that story to point maintainers to the problem, without triggering anyone's paranoia, or giving other state actors evidence of your auditing capabilities.

[matheusmoreira]

If a government is competent enough to detect this, they're competent enough to add it to their very own cyberweapon stockpile.

They wouldn't be able to do that for this particular exploit since it requires successfully decrypting data encrypted by the attacker's secret key. A zero day caused by an accidental bug though? There's no reason for them to eliminate the threat by disclosing it. They can patch their own systems and add yet another exploit to their hoard.

[Ajedi32]

> They can patch their own systems

"Their own systems" will necessarily include lots of civilian infrastructure. Hard to make sure all that gets patched without issuing a CVE, let alone without anyone in the general public even being aware of the patch.

[golergka]

> That said, if you're not using it, it defeats the purpose.

Not always. Weapons of war are most useful when you don't have to actually use them, because others know that you have it. This exploit could be used sparingly to boost a reputation of a state-level actor. Of course, other parties wouldn't know about this particular exploit, but they would see your cyber capabilities in the rare occasions where you decided to use it.

[kosh2]

> because others know that you have it.

Except that cyber weapons like these are

1. One time use 2. Expire upon detection (mostly)

I think this is simply just a tool for offensive action only.

[rigid]

> where no auditor ever looks

Well, software supply chains are a thing.

"where no auditor ever is paid to look" would be more correct.

[bandrami]

The purpose would presumably be to use this about an hour before the amphibious assault on $WHEREVER begins

[guenthert]

Hmmh, brings up the question, if no exploit actually occurred, was a crime committed? Can't the authors claim that they were testing how quickly the community of a thousand eyes would react, you know, for science?

[NekkoDroid]

That's like asking if someone that went into a crowded place with a full-automatic and started shooting at people but "purposefully missing" is just testing how fast law enforcement reacts, you know, for science.

After something like 2 years of planning this out and targeted changes this isn't something "just done for science".

[guenthert]

Or is it rather like someone posting a video on youtube on how to pick a common lock?

And what's about the fellows of U of Minnesota?

[tempay]

It’s more analogous to getting hired at the lock company and sabotaging the locks you assemble to be trivially pickible if you know the right trick.

The University of Minnesota case is an interesting one to compare to. I could imagine them being criminally liable but being given a lenient punishment. I wonder if the law will end up being amended to better cover this, if it isn’t already explicitly illegal.

[brokenmachine]

What happened at the University of Minnesota?

[tempay]

They intentionally tried to contribute security bugs into the linux kernel as a research project: https://lwn.net/Articles/854645/