[mindstab]

How did the attackers know what they were looking for. I'm going to assume that it's a small minority of linode users who have bitcoins on their machines. How were just these users targeted so accurately? What tied together knowledge they used bitcoins to those VMs and their linode accounts?

Also, was the nature of the attack just that the were able to login to your linode admin panel and from their root the machines and then loot your wallets?

[doublec]

They could have looked at the ip's of bitcoin services and noted those running on a linode. Or collected ip's from bitcoin nodes that their own node has seen and looked up those allocated to linode.

[darklajid]

>Also, was the nature of the attack just that the were able to login to your linode admin panel and from their root the machines and then loot your wallets?

The way I understand it the attackers were able to get access to the admin panel and invoked some kind of 'change root password' emergency stuff. The machines were rebooted it seems, which makes sense: The interface of Linode has probably/hopefully no access to the root password. Maybe this 'Reset my root' feature (now I'm guessing) reboots the machine in single user mode or passes init=/bin/sh to the kernel to reset the password once and reboots again afterwards.

Only THEN the attacker had access. But yes, he had root. The good (if you want to call it that) part of it is that this procedure rings every alarm possible. The real owner doesn't have the password anymore, as he'll soon figure out. It's everything but sneaky.

I DO wonder why root is allowed to log in at all, though..

[slig]

I disabled root login when I was setting up the server. Could my server be affected too?

Also admins that only log with ssh keys and don't use root won't be able to notice that, will they?

[devicenull]

Probably. You disabled root login how, via the sshd_config file? If so, you're still screwed.

Even if you fully disable root, that's not going to stop the init=/bin/sh script.

Even if you fix that (securing grub?) you're still screwed because it's a virtual machine, and they can just mount the partition to another VM, and pull all your data/reset root that way.

So, maybe if you have an encrypted partition, no root access, secure grub, and real hardware (it's probably possible to dump the VMs memory by snapshotting it, then pulling the key out that way), you would be secure against attacks like this.

With a VM? No, it's not nearly secure enough for very important things.

[darklajid]

Well, having the whole disk dm-crypted is kind of secure I guess. At least I still have no idea how I get at my ssl certification keys from startssl, although I have a dd of that drive in question from the vps provider. I was just too clever thinking of a long passphrase and too stupid to keep at least a hint around somewhere..

Total dataloss for me. But i fyou _do_ remember your dm_crypt password, I think you're safe against these kind of attacks

[darklajid]

1) No idea, that's something Linode needs to answer. I only guessed what it takes to change a root password of a VPS system.

2) Very good point. In that case it might work undetected for quite a while..

[draebek]

I would not be surprised if the Linode "reset root password" function shuts down your VM, mounts the filesystem directly on the host, and edits /etc/shadow, and maybe the PAM configs if they're feeling real nice. Using something we can't mount (e.g. encrypted)? Not using /etc/shadow nor PAM? Sorry, we can't help you beyond advising you try and reboot in single user and ssh in to your VM console.

[larrys]

"The real owner doesn't have the password anymore, as he'll soon figure out."

He won't figure it out until he tries to login though.

[3pt14159]

Simplest answer is probably the right one in this case:

Someone at Linode did it. Ran a script to see how many bitcoin files there were on all the machines (they probably do these types of queries for anti-virus/whatever anyways) and took a customer support password to log in and get the coins. If he did it right he still might be working there, as it is easy to get credentials from friends/coworkers (even though it should be really really hard).

[lawnchair_larry]

Sorry, there is just no way that this is the case. Please don't throw such a serious allegation out there without any evidence. To even suggest that this is technically possible for an employee to do is a serious allegation, let alone suggesting that someone did it maliciously. This spreads all kinds of FUD.

I'll happily eat my words if that turns out to be what happened, but it is definitely not the simplest answer.

[RLG_RLG]

Inside job is usually the answer for targeted attacks against inside systems. Inside collusion at a minimum.

I wonder how anyone can trust their linode systems after an admin account being compromised.

It would likely ruin their business to re-install everything, but that is the only way to know root kits have not been installed.

[lawnchair_larry]

Or just check if your instance was restarted and your root password was changed. If it wasn't, you were not exploited this way.

[shadowmint]

Up-vote to the original comment because it's not stupid or impossible, just unlikely.

We can only speculate at this point.

The simplest answer is probably that one of the staff was subject to a targeted hack and a 3rd party gained external access to the CSR tools.

Possibly for an extended period of time. <-- This is the concerning part.

It's relatively unlikely an internal staff member would do something this dumb (but, not impossible. we've had this happen _here_ where I work, with credit card numbers, but obviously the person responsible was caught almost immediately).

[ohgodthecat]

> To even suggest that this is technically possible for an employee to do is a serious allegation

It is technically possible for an employee to do it because it seems (from the linked pastebin above) that is how it was compromised, an elevated account for linode manage was compromised.

As for an employee being the one that did it, that is probably the least likely cause.

[showerst]

I believe he's referring to the part about employees (at least the ones that have access to the customer dashboards) being able to run a script to scan for bitcoins.

[nknight]

You can't secure against God. Of course it's technically possible for a Linode employee to do.

[trotsky]

Pretty sure there is a default port that accepts connections as part of bitcoind, so you can just portscan for it.

[ajross]

I'd be very surprised (and suspicious) if they were running any kind of diangostics over their customer's data without an explicit signed contract. The liability worry there alone is scary.

[gravitronic]

you do realize that in this theory, the same person then went and stole thousands of dollars in bitcoins, right? I don't think they were worrying about liability...

[ajross]

You misunderstand. Not the thief's liability, Linodes. If someone, say, engages in insider trading because of something they saw in Linode's own analysis system, Linode can be sued for failing to protect that information. If they have a policy of never reading customer data (and can prove it) that becomes much harder. The posited "anti virus checker" would throw that promise out the window.

[Drbble]

How so? Google credibly claims (somehow) that they don't "read" customer email, even though they run the largest-scale automated email reading system on the planet.