Hacker News new | ask | show | jobs
by bsamuels 1512 days ago
Every tech company is losing the war against credential stuffing. I have a friend working at a series B startup with <10k MAU, you wanna know how many login attempts there are each month? 25,000 login attempts. Per user. That's 250m login attempts each month using stolen credentials.

None of the service providers who claim to fix the issue are worth their weight in salt. Shape, Akamai, none of them have a grip on the problem because the attackers are constantly evolving. As you can see, even Google is capitulating despite all the fud that people on HN spread about the company being omniscient.

Anyone who thinks this is about advertising/collecting personal data is out of their minds.

The worst part is nobody can talk about it because anything you reveal about your problems can give the attackers a massive edge.

7 comments

There are far better ways to stop credential stuffing than requiring a phone number that would be immediately obvious to the people at Google - Hashcash, for instance[1].

250M login attempts times a few seconds of CPU time is a lot of compute cost to inflict on an attacker who is carrying out the same attack against a bunch of other services at once, and virtually nothing to the few thousands of active users who should only be logging in once every few months each.

And yes, a few extra seconds of logon time is viable, because people are used to the login process taking a few seconds and they don't do it very frequently.

"Credential stuffing" is straight-up an invalid excuse for asking for someone's phone number.

[1] https://en.wikipedia.org/wiki/Hashcash

Attackers are using hacked IoT devices to do these attacks. These devices have roughly the same computing power as a mid level smartphone. Attackers do not use their own hardware, and don't care about how much energy is used by their bot devices.

In a normal attack, there are maybe 2-3 requests per hour that come from each hacker-owned device. The only thing that hashcat would do is drastically increase power consumption at no cost to the attacker, and turn the application into a battery drainer on mobile devices.

So no, Hashcat is not an adequate solution.

> Attackers are using hacked IoT devices to do these attacks. These devices have roughly the same computing power as a mid level smartphone.

False for a very large variety of low-power IoT devices using chips like the ESP32, which are multiple decimal orders of magnitude slower than a modern computer (or high-end smartphone) and will absolutely take far longer to compute a Hashcash challenge than one of those devices. Your smart light bulb is absolutely not hosting a Qualcomm Snapdragon 450 to change its colors.

Additionally, adding this compute+power load causes the presence of malware to become far more visible on these devices, which increases risk of discovery, which is a significant upside.

Finally, because there's now a significant computational cost to performing a login attempt, the value of individual devices are further decreased to the attacker, which reduces their likelihood of compromising these devices, and for using them for these kinds of attacks against services that use this measure.

So, yes, Hashcash is absolutely an adequate solution, and yes, there is absolutely a cost to the attacker.

You have no idea what you're talking about. Botnets are almost entirely ISP router/modem combo devices.

Hashcat was proposed over 20 years ago. You really think out of all the tens of thousands of security engineers working on this problem, nobody has ever considered it? Get a grip.

I hate how this website incentivizes people to try to make posts that sound smart instead of posting stuff they're actually knowledgeable about.

> Botnets are almost entirely ISP router/modem combo devices.

Above you say they are IoT devices. I don't mean 'gotcha', but to learn: What does the population of botnet devices consist of?

This comment violates probably about a dozen different parts of the HN guidelines[1]. You should calm down with the ad-hominems and substance-less claims and give the guidelines a read, then come back and make an actual argument.

[1] https://news.ycombinator.com/newsguidelines.html

Yeah the majority of them aren't Wifi lightbulbs afaik it's mostly routers and other similar devices, so they really do have the power of a low-mid range smartphone.

Realistically though as long as it can send a request I think attackers would prefer lower power devices someone's computer may be able to send many more r/s but much harder to gain control of versus the $30 iot device.

This is ab irrelevant tangent on the mid level phone comment. We all know you don’t need any compute power at all to make “2-3 login attempts per hour” even on the slowest of IOT devices.
The problem with proof-of-work-for-login is:

Some of your attackers are going to run your proof-of-work algorithm on a 3090 Ti GPU and put loads of work into optimising their setup.

Some of your legitimate users are going to run it on a Raspberry Pi 1 with an ancient browser that only runs wasm through a javascript polyfill.

Tough to make up for a 1000x performance difference.

3090 Ti's are very hard to come by - an attacker will need to either have malware on gaming rigs (which are user-facing devices and so have higher odds of detection, especially if you're loading those GPUs), be paying for cloud compute (money out of their pockets), or own the GPU themselves (more money out of their pockets) - so that's a significant cost imposed on them - which is greatly amplified given the above figure of 25k login attempts per month per user per service.

Those legitimate users logging in on a Raspberry Pi 1 are going to be in the vast minority (probably about 0.001%, if not less, as evidenced by the relative marketshares of Windows vs Linux, and how rare Raspberry Pi's are for desktop use relative to normal x86 machines) of users of your service - it's OK for them to have 20s-30s logon times in exchange for (1) not requiring PII from users and (2) less credential stuffing.

Certainly, Raspberry Pi users are more rare than Internet Explorer users, which many developers simply refuse to support at all.

In particular, I am ok with having 30s logon times for services for my desktop computers (assuming that I only have to log on once every few months and not every few minutes like my brain-dead bank requires), so it's more than reasonable to expect those with an extremely niche and underpowered setup to have to wait that long for a very infrequent login process - again, for the sake of security and privacy.

Then give those users an option. It's not hard. Show a little progress bar for the PoW, and then offer the user other options including manually approving the new logon and using a yubikey or similar.
Mostly, there are so many IPs those attackers can use. When I look at the smtp/imap failed authentication attempt logs, the patterns are fairly obvious. Different IPs trying similar looking emails in a sequences. I think in most cases where banning the IP, even only for 24h, will do the trick (and /64 block for v6).

Also there is something to be said for generating the password yourself and sending it in clear to your users by email. The reality is that if the attacker has access to your client’s emails, it’s game over anyway because of password recovery. And this way you enforce that your clients will not reuse a password, and will have a strong non brute forceable password. And if you get hacked, at least you didn’t leak that precious password your users reused everywhere else. The only issue I can think of is that’s because smtp is the most neglected protocol of the internet, there is no way to ensure the email will be encrypted in transit.

Attacks coming from the same IP address are literally kids just running port scanners.

Blocking individual IP addresses has never been a valid mitigation against professional attackers. Attackers will just pivot to renting a different botnet from different geographies/ip ranges.

If your service has any level of scale, there can be double digit percentages of users who are sharing an IP address with a hacked device. Blocking attacker IP addresses will block users of your service.

Proof of work rewards those who own fast CPUs (or are using stolen CPU time). Those who are using an non-high end Android phone, a battery powered laptop, or an older PC are punished with a long loading times. Part of why Gmail takes so long to load is the anti-bot scripts.

Use rate limiting instead.

> Proof of work rewards those who own fast CPUs (or are using stolen CPU time).

Most technology rewards people who own fast CPUs or are using stolen CPU time, so this statement doesn't mean much.

What actually matters is "does this significantly negatively impact those with slower processors" (no) and "is it still effective at punishing attackers" (yes).

Impact: I'll set my Hashcash challenge so that it takes about 8 seconds on a 10-year-old middle-of-the-line Intel processor. There, login time is about doubled for people running that extremely old hardware, and that's the worst case - and this is entirely acceptable given that most people do not log in to their accounts very often - they do so once and then stay logged in for weeks or months.

Effectiveness: Even for attackers using stolen CPU time - Hashcash is still effective. If you have malware on a low-power device, that device is going to be far slower at login attempts. If you have a user-facing device, they're much more likely to notice their computer being slow, and do something that might lead to discovery. If you've stolen compute on some cloud, then the IT folks are more likely to notice, and so on.

Rate limiting is good, but is complementary to proof-of-work - if you have a large pool of IPs/devices to log in from, for instance, rate limiting isn't very effective.

> Anyone who thinks this is about advertising/collecting personal data is out of their minds.

Sorry, but that trust has been burned and I don't see a path to recovery. Support hardware tokens or get off my lawn.

https://www.eff.org/deeplinks/2019/10/twitter-uninentionally... https://techcrunch.com/2018/09/27/yes-facebook-is-using-your...

the average person don't have any idea what's a hardware token. google is not catering to HN readers, they're catering to your grandma, your parents, your little brother/sister, your tech illiterate neighbor.

They couldn't care less about the habits of a nerd on archlinux with ublock, noscript, firefork a vpn, hardware tokens and 2FA everywhere with recovery code split in 7 different location.

This is a straw-man. The problem is not that Google is designing their services to cater to the average tech-illiterate user, it's that they're preventing the tech-literate users from opting out of phone recovery and/or using something more sane, like what's been listed above.

That's clearly malice. Like, there's no good reason that Google would require you to hand over a phone number.

> That's clearly malice. Like, there's no good reason that Google would require you to hand over a phone number.

Let me give you a reasonable non-malicious reason:

Googler A: "We have this new attack. People are creating accounts from compromised IPs, and then creating app passwords to send huge amounts of gmail spam through SMTP directly, thus avoiding our browser-based spam mechanisms"

Googler B: "Can we ban them?"

A: "We can't ban them because we have no info on them, just sign-up IP, and the botnet has practically unlimited IPs"

B: "What about forcing them to have a phone number so we can do anti-spam on that, and perma-ban compromised phone numbers from making new accounts?"

A: "Good idea, that'll stop such a huge quantity of phishing emails and spam. That'll be good for the internet as a whole"

----

See, a non-malicious explanation.

Pretty sure they do support hardware tokens.
Only after you give them a phone number. In fact, they allow you to remove the phone number afterwards, so clearly they're happy with non-SMS 2FA being the only 2FA method on the account, as long as they first get the opportunity to stalk you beforehand.
Do they accept burner phone #s?
There's no universal definition of a burner phone number, but they do ban certain number ranges commonly associated with VoIP providers. Your best bet is to get a prepaid SIM as those typically draw from the main number pool of the carrier so scum like Google can't ban those without also banning a third of their target market.
> so scum like Google

You do know it isnt mandatory to be a customer.

+1..get a yubikey or similar device, problem solved.
> Every tech company is losing the war against credential stuffing. I have a friend working at a series B startup with <10k MAU, you wanna know how many login attempts there are each month? 25,000 login attempts. Per user. That's 250m login attempts each month using stolen credentials.

I ran into a similar situation (small, growing startup dealing with credential stuffing attacks). We have since implemented a few different solutions, but one of the most successful was rejecting reused passwords at signup using this service [0].

Some other effective solutions include captchas, emailing a verification code, etc. Aggressive rate limiting was not at all successful, as the botnets seem to have endless piles of residential ip addresses to send requests from.

[0] https://haveibeenpwned.com/Passwords

This. This reminds me of a couple of other hot threads on HN like Google not allowing you to log back in to old dormant accounts. It's abundantly clear to me that it's the same thing, same motivation.

The world desperately needs to move away from username + passwords. Google might seem really pushy right now, but it's simply a trailblazer. My prediction: in a few years every company that handles valuable personal information will behave like what Google is doing today.

at 25,000 login attempts per user if you're not doing common sense rate-limiting of attempts per username, and rate limiting per IP space origin (either discrete ipv4 /32 or attempts from within a whole ASN), you've got other problems.

the rate-limit and blockage time for attempts should increase ban time/lockout-timer on an exponential time scale the more that a single browser/useragent/browser fingerprint/IP makes incorrect attempts.

yes obviously there are people out there with fully automated systems who will try massive lists of commonly used plaintext passwords for authentication if you don't throttle/rate-limit it.

>yes obviously there are people out there with fully automated systems who will try massive lists of commonly used plaintext passwords for authentication if you don't throttle/rate-limit it.

and those people use single browser/single useragent/single browser fingerprint/single IP

the people competent enough to send millions of requests are usually also competent to send hard to detect requests

there are dozens of (free)tools/services offering those capabilities for LEGITIMATE purposes(like scraping)

there is an even bigger underworld market for paid tools for illegitimate purposes

if you're sending millions of requests you still have a finite number of proxies to use, it would be thousands to tens of thousands of requests per discrete /32 ipv4 proxy address, and not hard to detect as an abnormal volume of attempts per IP.

even if you see something like a single /32 address that is probably the public facing endpoint of a mobile phone carrier's cgnat and has MANY users behind it, trying different password attempts, you can still rate limit the number of attempts per unique username.

the actual amount of legit requests that a human who has forgot their password makes is 99.9% of the time under ten requests per account before they give up.

And that "finite number" is tens of millions, or even hundred of millions, spread across providers, spread across almost any location. From 4G proxy farms, to botnets of residential IPs, to grey area apps that rewards user for sharing their connection.

And ok, let's assume it came from the same block of ips(which rarely happens) what do you do when those IP blocks are from IDK, Verizon or AT&T in the middle of New York?You block half the city?

+if the attacker is trying it on all accounts, what are you gonna do? rate limit all accounts? and now anytime a user forgot his password he have to contact support because he can't even do it himself so your support is overwhelmed by 1000s of request every day?

> Anyone who thinks this is about advertising/collecting personal data is out of their minds.

For the end user it is absolutely about collecting additional personal data. My concerns as a user are not the concerns of Alphabet, the company, period.

> Anyone who thinks this is about advertising/collecting personal data is out of their minds.

Google is a public company, just look at their annual reports. Where do they make most revenue? Advertising.

Thus, everything they do is about promoting advertising. It would be naive to think otherwise.

And how do they avoid wasting money and therefore increase their profits? By putting in automated systems, that in this case are good enough for the majority of non tech savvy, non paranoid users.