[gizmo686]

From the mail:

>This patch solves a problem where simultaneous reads to /dev/urandom can cause two processes on different processors to get the same value. We're not using a spinlock around the random generation loop because this will be a huge hit to preempt latency. So instead we just use a mutex around random_read and urandom_read. Yeah, it's not as efficient in the case of contention, if an application is calling /dev/urandom a huge amount, it's there's something really misdesigned with it, and we don't want to optimize for stupid applications.

[helper]

I guess that means all go applications that use crypto/rand are considered misdesigned then[1].

[1]: http://golang.org/src/pkg/crypto/rand/rand_unix.go#L30

[jerf]

If you're using crypto/rand to yank a whole bunch of random numbers out for the purpose of deciding which DNS record to use when multiple DNS records were returned, yes, the Go application is misdesigned. Such applications should be using math/rand. Seeding your math/rand from crypto/rand isn't a bad idea, but you don't need to be hammering on /dev/urandom in such code.

[throwaway2048]

This "some random data is more important then other random data" musical chair dance going on with /dev/random vs /dev/urandom vs userland [CS]PRNGs (often gathering from extremely poor sources, or using broken algos) has been nothing short of an unmitigated security and useability disaster.

We have the abillity to make the /dev/urandom CSPRNG secure enough and fast enough for (almost) any randomness purpose. We need to cut all the rest of this insane crap.

People choose the wrong RNGs and get burned, or wont use the right ones because of speed or imaginary entropy exhaustion issues. This matters.

[acqq]

It's impossible (or just not worth the trade-off) to make one piece of software (this time, kernel) fast for any use case imaginable. In this case, kernel behaved correctly but with the speed degradation for extreme cases. That the author's Rube Goldberg machine then runs slow I don't consider kernel to be guilty.

The guy uses PHP and instead of built-in HTTPRequest he uses curl to make a request to "a bucketed key-value store built on PostgreSQL that speaks HTTP which uses Clojure and the Compojure web framework to provide a REST interface over HTTP." A bit of shooting the flies with cannons on every side?

On another side, if it can be proved that urandom has serious problems in reasonable use cases it should be checked what can be changed and how.

[nubs]

Slightly off topic, but I wanted to clear this up:

> The guy uses PHP and instead of built-in HTTPRequest he uses curl to make a request

HTTPRequest is not built-in to PHP. It is a PECL extension that is usually installed separately from PHP.

Curl is more built-in to PHP - it's a PHP compile-time flag, and it is distributed with PHP source.

[acqq]

So the best approach for the given problem (just send the request, fetch the response, without too much overhead) seems to be using something a bit lower level:

http://at2.php.net/stream_socket_client

[dlitz]

> It's impossible (or just not worth the trade-off) to make one piece of software (this time, kernel) fast for any use case imaginable.

We're not nearly at the theoretical limit of what /dev/urandom can provide.

[dlitz]

Exactly. Most developers are about as good at picking RNGs as end-users are at picking passwords. We need to stop asking them to make that choice.

[tptacek]

Yes. Exactly this.

[helper]

There are lots of applications that require a cryptographically secure random number generator, which math/rand doesn't claim to be.

[jerf]

This is not one of them.

[azinman2]

And while that's interesting from the perspective of fetching web content via curl (and kudos to the author tracing it down to that), that doesn't mean the fundamental issue shouldn't be fixed.

In the current security environment, from heartbleed to the NSA, it's becoming clear that security issues need to be systematically dealt with from an industry perspective or people will start to lose faith in secure Internet communication, which would undermine too much of what's valuable about the Internet.

What we need is great APIs/frameworks/design patterns to simplify cryptography so that a newbie ruby on rails programmer CAN create actually secure applications and not even realize that it was complicated in the first place.

In cryptography you make one misstep and the entire chain is broken. It's thus important for things like the linux kernel to provide great implementations so that people don't think twice about using it and never want their own PRNG.

[acqq]

Why does kernel have to be "fixed" if anything in the current "APIs/frameworks/deseign patterns" is not doing its own homework?

It seems the author admits in the comments: "All the application needs to do is open a socket and generate a GET request." So why complaining about the kernel?

If there's problem with urandom, demonstrate it on the reasonable use case example, don't try to impress anybody by showing how much different libraries, modules and programs you combine for one key-value query.

[ekimekim]

Are you also suggesting that the kernel provide a "great implementation" of SHA? Of SSL? Of https? Where do you draw the line?

There's no reason for the kernel to provide standard library functions. In fact, I'd argue that syscalls should be reserved for only actions that cannot be done wholly in userspace (futex is a good example of this). The current model of "hardware randomness to seed a PRNG" makes sense. It is up to the userspace libraries to provide good implementations.

[mikeash]

You don't need to be, but why not? It should be plenty fast and work well. If it's turning out to be too slow due to too much locking, that should be fixed.

[raverbashing]

1 - Rand is faster

2 - You don't need the crypto qualities of it and you're emptying the entropy pool for nothing

3 - You're doing much more work, especially if you're reading one byte at a time from /dev/urandom (doing a syscall, etc), while rand is just a calculation

[tptacek]

There is in practice no such thing as "entropy depletion". The retail side of a CSPRNG is very similar to a stream cipher. The idea behind "entropy depletion" is structurally the same as the idea of a stream cipher "depleting its key". You can run AES-CTR as a stream cipher for several exbibytes before the output starts becoming distinguishable (which is not the same thing as "reveals the key").

[raverbashing]

True, unfortunately /dev/random blocking "soon" in Linux helps to propagate this myth. I stand corrected.

[mikeash]

1 and 3 are the same thing. I think the best way to address these, if performance is a problem (don't optimize what doesn't need it) is buffering to reduce syscalls, and optimizing the kernel implementation to fix the sort of internal performance problems like the link describes.

For 2, entropy pool depletion is a fictitious problem if you're worried about security. Some discussion here:

https://news.ycombinator.com/item?id=7361694

If you're worried about blocking apps that use /dev/random, the answer there is to fix them to use /dev/urandom so they don't block.

[jerf]

Yes, it should be fixed. Yes, it's still a "misdesign" to use the cryptographic random number generator when you just want "a" psuedo-random number, right now. For choosing which of the several DNS answers you use, you could pretty much get away with keeping a counter and returning that counter modulo the number of choices. It's technically wrong for several reasons, but you could get away with it. That's how low-impact this random number usage is. Using a cryptographically secure random number generator for that is always going to be overkill for such a task.

[zatkin]

So, would a possible solution be to check how many people are using the random generator at once? If only one process is currently accessing /dev/urandom, then avoid the spinlock and problem solved. Or, I am completely wrong.

[mmckeen]

If there is only one process, then no spinlock contention, so no problem.

The problem comes with multiple processes competing for the lock.

[ef47d35620c1]

Would a possible solution be to check how many people are using the random generator...?

One preson may have multiple processes reading from /dev/urandom.