[londons_explore]

I suspect I could have used this exact attack against 10,000 random SSH servers spread all over the world, and not be detected.

Most people don't log TCP connections, and those that do don't go through their logs looking for odd certificates in ssh connections.

And no common logging at the ssh/pam level would have picked this up.

Your only chance is some sysadmin who has put 'tripwires' on certain syscalls like system(), fork() or mmap() looking for anything unusual.

Even then, they might detect the attack, yet have no chance at actually finding how the malicious code loaded itself.

[amscanne]

There is no ‘system()’ syscall, and fork/exec would be extremely common for opensshd — it’s what it does to spawn new shells which go on to do anything.

I’m not arguing with the point, but this is a great place to hide — very difficult to have meaningful detection rules even for a sophisticated sysadmin.

[ivlad]

This would be execve() that did not go through PAM dance and end up being privileged process.

I _think_ it’ll look very different in ps —-forest output.

[amscanne]

It’s true that there’s a precise set of circumstances that would be different for the RCE (the lack of a PAM dance prior, same process group & session, no allocation of a pseudo-terminal, etc.). My point was merely that I don’t think they are commonly encoded in rule sets or detection systems.

It’s certainly possible, but my guess is sshd is likely to have a lot of open policy. I’m really curious if someone knows different and there are hard detection for those things. (Either way, I bet there will be in the future!)

[ivlad]

I am trying to figure out if auditctl is expressive enough to catch unexpected execve() from sshd: basically anything other than /usr/bin/sshd (for privsep) executed with auid=-1 should be suspicious.

[matrix_overload]

With sufficient data points, you can do A/B and see that all affected systems run a specific version of Linux distro, and eventually track it down to a particular package.

[bastawhiz]

Unless you're the bad actor, you have no way to trigger the exploit, so you can't really do an a/b test. You can only confirm which versions of which distros are vulnerable. And that assumes you have sufficient instrumentation in place to know the exploit has been triggered.

Even then, who actually has a massive fleet of publicly exposed servers all running a mix of distros/versions? You might run a small handful of distros, but I suspect anyone running a fleet large enough to actually collect a substantial amount of data probably also has tools to upgrade the whole fleet (or at least large swaths) in one go. Certainly there are companies where updates are the wild west, but the odds that they're all accessible to and controllable by a single motivated individual who can detect the exploit is essentially zero.

[guenthert]

There are those who run sshd on a non-standard port and log all attempts to connect to the standard port though.

[Hackbraten]

Those connection attempts wouldn't ever reach the daemon though, let alone get to preauth. So how would an exploitation attempt even be distinguishable from, say, a harmless random password guess if neither ever gets to see the daemon?