A malicious process allocates a 256 member array. Then it creates a conditional where the speculatively executed part writes a byte at offset array + kernel_memory_value. The speculative branch is executed but then backed out, but the byte in the array was touched so it is in cache now. Then the malicious process reads all of the members of the array and looks for one that returns much faster than expected (is in cache) and they know the value of that byte. Rinse and repeat to read the rest of the kernel memory. It's not going to give you MB/s of throughput, but it's plenty fast to read some key material or process tables or anything like that.
It's a very impressive attack. My hat is off to whomever thought it up.
I didn't read it as whether whether the attack is impractical (because it is clearly quite practical) -- the parent was questioning whether it's practical (or not) that such an attack would be "planted" as a backdoor by an agency like the NSA. The attack comes off as quite impractical for something like a plant (sensitive to e.g. compiler output, requires locally executing code to snoop is already a red flag, and the 'bug' enabling this has really been considered a feature of every mainstream CPU for like 15 years, and not considered by many to be any kind of attack vector.)
Or maybe the idea is speculative execution itself was a dream of the NSA that was Inception-planted into the brains of CPU designers in the 90s; who knows what the theory-of-the-hour is regarding 3-letter-agencies and their capabilities.
Ultimately I think what we're really learning is that guarding against things like microarchitectural attacks on contemporary superscalar, OoO CPUs is going to be an uphill battle that we didn't ever think of due to incidental complexity (among other reasons), and will serve as a new class of attacks. Who knows how long this bug class will exist; we've killed some. What's also likely is that, like most security failures in the industry, this is a result of various things like basic lack of forethought/ill considered design, as opposed to plants (3 letter agencies aren't responsible for the vast majority of security failures you see, it's simple mistakes). But peddling conspiracy theories involving them gets you upvotes, so, you know...
The Meltdown paper cites 500 KB/s average throughput when transactional memory extensions are available on the Intel CPU. It's not MB/s, but it's still pretty fast.
So it speculatively indexes into the array by the value stored at that memory address and writes the byte. Then to figure out what the value is you just have to see which element in the array is cached.
To elaborate on this, the write to the array isn't what's being read here.
array[value_of_kernel_memory_byte] = 1;
This assignment gets rolled back like it's supposed to. It's when reading the array after the rollback that the exploit measures that a read to array[value_of_kernel_memory_byte] is faster than the rest because that index is already in the cache.
I replied to a comment saying "Or NSA had them do this intentionally"; my comment is not about whether this is a vulnerability, or whether exploitation is viable (it absolutely is), but rather about whether this is a viable backdoor.