[VonLipwig]

The message seems to be...

Only 8 accounts were affected. Do not worry. Minor breach. Not much harm done.

It seems to me the truth is the attacker looked for bitcoin wallets and emptied them. The fact he could identify 8 accounts and access them suggests the attacker could have accessed far more accounts if they wished. I think this is the most worrying thing about the breach.

I don't really understand how bitcoin works but it seems that people with wallets need to set up multiple wallets on multiple providers and limit the amount of bit coins in each wallet to limit any losses from breaches like this.

If I was a linode customer I would be thinking about moving. This message, while fairly open, doesn't give me much confidence there aren't other security issues with the platform.

[drostie]

Just imagine that BitCoin is like having cash in your wallet, because that's more or less its intended model. There are a lot of 'anti-counterfeiting' measures because computers are very good at copying, and you don't want people to be able to copy BitCoins the way they can copy music -- and when you ask "what is BitCoin?" people basically start to tell you about the anticounterfeiting technology, and the limits on printing uncontrollable amounts of money. But it's essentially stamped paper in your wallet in any other sense, worth whatever people using it on the Internet will pay for it, not backed by anything in particular but its usefulness.

Basically a lot of people were renting storage rooms in an apartment complex run by Linode, you get your own key to enter the door and retrieve and store things -- whatever. Some people left their wallets inside these buildings, with cash therein. Someone else used some unidentified systematic security flaw, but we don't yet know what it was. Maybe there is a ventilation system which is easily navigable once you know how to get in; or maybe all of the rooms have unlocked windows for no good reason; we haven't been told yet. (There are some suggestions that they stole a key from one of the janitors who cleans these rooms up.)

What we have been told is that some burglar stole eight wallets, and that "All activity by the intruder was limited to a total of eight customers, all of which had references to 'bitcoin'." That suggests that the burglar did indeed peek in the windows beforehand somehow, to find out that these 8 rooms had wallets inside. Otherwise, presumably they would say something like, "The intruder broke into many of our customers' accounts but didn't actually do anything in 99% of cases." In that sense I think the scary bit isn't that he accessed the 8 accounts, it is the fact that he identified them in the first place.

Amortizing the loss across many points of failure may be a good idea, but it wouldn't seem to solve the central problem. Suppose I put $20 in two accounts with 5% chance of compromise, rather than $40 in one account with 5% chance of compromise -- either way, I should expect to lose $2. What I've changed is that I am more likely to lose some of my money (9.75%), but I am less likely to lose all of my money (0.25%). This may appeal more to risk-averse people but it is not fundamentally changing the situation.

Perhaps a better approach is to keep a BitCoin wallet encrypted, since that's pretty simple to do in day-to-day life. This is something that you can't do with your wallet -- you cannot turn your wallet into a steel vault with two-foot-thick walls.

[lambda]

> In that sense I think the scary bit isn't that he accessed the 8 accounts, it is the fact that he identified them in the first place.

This isn't all that surprising. There are basically two reasons why you would have a Bitcoin wallet on a server: if you are mining using the CPU power of that machine, or if you need to send Bitcoins from an online application. For example, one of the people who mentioned having coins stolen was from a mining pool; you need some automated system to pay out the earnings to the people who have been doing mining, and so the wallet for that automated system was on the server, and was stolen. I suppose one further reason might be as a backup, but in that case, I dearly hope that it's an encrypted backup without the encryption keys in the sever.

Given these reasons for having your wallet on the server, it's not surprising that people found them. These require network-facing services, that are easy to trace back to the server in question. The mining pool is a public service; anyone can join, and find the address of the server. Furthermore, when you make payments, you announce them to the full Bitcoin network. Someone sniffing transactions can watch where transactions originate, and target that. If they already had a compromised customer service account on Linode, they probably watched the Bitcoin network for a while, made note of transactions originating from IP addresses in the Linode range, and then targeted those accounts.

One way to protect yourself from this would be to proxy your Bitcoin transactions through a host other than the one that has the wallet, obfuscating where the transactions are actually coming from. You could even go so far as to make all of your transactions via Tor, which would probably make it fairly difficult to find where your Bitcoin wallet actually lives.

> Perhaps a better approach is to keep a BitCoin wallet encrypted, since that's pretty simple to do in day-to-day life. This is something that you can't do with your wallet -- you cannot turn your wallet into a steel vault with two-foot-thick walls.

The problem is, if you need to make payouts from your wallet, then the machine that does that needs to be able to decrypt the wallet. That machine can then be compromised to be able to steal your keys. Encryption doesn't buy you all that much, unless you are just doing a backup and don't need the machine to be able to do online transactions at all.

Perhaps another solution would be to encrypt each key in your wallet separately using a k-out-of-n encryption scheme (where produce n keys, any k of which can decrypt the wallet). You can then distribute those keys to independent hosts, which hopefully should not all be subject to the same vulnerabilities. Then any time you do a transaction, k of those hosts will need to produce their key to decrypt the key in your wallet and perform the transaction. That way you would have to compromise several different, independent hosts in order to steal the wallet.

Of course, this would drastically increase the cost and complexity of the system; and you would need to ensure that whatever system that authorized payments was likewise distributed, which if you had, say, a web-facing service would be difficult.

The easiest thing to do to reduce the risk is to only leave enough value in the wallets that are on the servers for a couple days worth of transactions. Then you transfer Bitcoins from a more secure location once a day to keep the coffers full. This is not much different than a physical store; yes, you are at risk of being robbed, but if you only have one days worth of cash there, with the rest somewhere more secure, you reduce how much risk you have.

[j_s]

If they had used 'unlock-at-boot'/true-crypt style disk encryption and kept the password/key off the machine they would have been safe from this attack. (They would also have to provide the password/key every restart.) It is only in hindsight that something like this seems worth implementing!

Your more general solutions would protect from an attack that rooted a live system rather than just resetting the root password while the machine was offline.

[microtonal]

Indeed, we are a Linode customer, and this message only helps a bit. Yes, I know now that we are not affected. But little other information is given: were the user accounts compromised by a vulnerability in Linode's VM management software? If so, was this vulnerability found and fixed? Or did the attacker compromise the account of one of Linode's employees?

[epaik]

I believe in an earlier statement they did say that they fixed the vulnerability. No idea why they didn't explicitly state so in this new statement.

[StavrosK]

I don't think there was a vulnerability. As I understood it, somebody stole a support person's credentials and logged in with them.

[lambda]

Yes, but how did they steal the credentials? Did they find a sticky note with the username and password written down? Or was there a vulnerability in the admin interface that allowed someone to sniff credentials? Or did they hack into the personal computer of someone with admin privileges?

Basically, this announcement gives me no confidence that they've done due diligence in fixing this problem. They haven't explained what the vulnerability actually was, nor what they have done to avoid it in the future.

Of course, this does speak to the dangers of using hosted services for anything that needs a high level of security. Anyone with appropriate admin privileges on the host system can compromise any user. That increases the attack area considerably; you don't need to attack the system directly, nor the users of the system in question, you just need to find one person who has admin privileges who is vulnerable, steal their credentials, then attack any users at your leisure.

[microtonal]

Ok, I missed the earlier statement. Thanks for the information!

[mdda]

It wasn't clear from the memo, but how did the attacker know which 8 specific accounts had 'references to bitcoin' if they didn't access other accounts too?

[calloc]

Are you able to name your VM's on Linode?

[ukd1]

I doubt it's that: I would have worked out the target sites hosts (looking up the IP), made a list, worked out the most popular by value - probably worked out it was linode and gone from there...once in, I'd bet it's a simple task of searching for the IP in their interface.

[mdda]

Ok, asking the other way around, did Linode check out the non-attacked VPSs to determine whether they didn't have bitcoin wallets on them?

I'm not disagreeing with anything else said here - it's just that it seems that the memo was very conclusive in stating something that couldn't be known unless there was more VPS introspection than they claim...