I’m not a fan of Extropic, but I’m seeing a lot of misconceptions here.
They’re not building “a better rng”- they’re building a way to bake probabilistic models into hardware and then run inference on them using random fluctuations. Theoretically this means much faster inference for things like PGMs.
Skimmed the litepaper. Has the flavor of: you can do "simulated" annealing by literally annealing. I like the idea of using raw physics as a "hardware" accelerator, i.e. analog computing. fwiw, quantum computing can be seen as a form of analog computing.
I do think that a "better rng" can be interesting and useful in and of itself.
Thanks for the Normal Computing post, it felt more substantial.
We experimented with doing ML training with it, but it's not clear that it trains any better than a non-broken PRNG. It might be fun to feed the output into stable diffusion and see how cool the pictures are, though.
Cloud RNG number streaming is interesting but costly, no? I did have the idea to serve truly random numbers via a quantum computer (trivial by just preparing the simplest state and measuring). Anything else can't be said to be truly random.
You don't need a quantum computer to sample noise from quantum processes.
It's a prohibitively expensive way to go, and depending on how you built the quantum computer, it may be more susceptible to interference and non-quantum noise than using good circuits and custom systems.
Oh sure, it was more of a joke app idea, or in another sense, for those who want philosophically "perfect" randomness. After all, assuming a single bit sample, anything but a hadamard state has nonoptimal entropy. Of course, it makes almost no sense to sample a bit at a time since we want pragmatism. But having a virtual coin flip essentially "create" a new, nonexistent bit of information in the universe -- that's funny (well, assuming certain interpretations of QM).
I'm using the term "analog computing" to mean using non-digital (or even digital but in a nonstandard way). A quantum processor is not digital as each qubit has an uncountable number of states. A quantum computer would likely have a classical (digital) part to measure the quantum processor's registers, which are a bunch of "analog" states. And even if one wants to use terminology in a way that qc isn't analog, it would still objectively share many qualities with "normal" analog computing (basically all of its differentiators against classical compute).
Then again, I should probably also ask what you mean by "analog computing", and why you think quantum error correction would not allow qc to be classified as analog.
For context, I did research on crafting a high-fidelity (error-correcting) quantum gate (successfully).
It did make me curious however, if we dropped the requirement that operations return correct values in favor of probably correct values - would we see any material computing gains in hardware? Large neural models are intrinsically error correcting and stochastic.
I’m unfortunately not familiar enough with hardware to weigh in.
The trouble is if you use actual randomness then you lose repeatability which is an incredibly useful property of computers. Have fun debugging that!
What you want is low precision with stochastic rounding. Graphcore's IPUs have that and it's a really great feature. It lets you use really low precision number formats but effectively "dithers" the error. Same thing as dithering images or noise shaping audio.
Yeah debugging would be a pain, but in the context of inference/training unnecessary. There is some set of ops which requires high precision, if I L2 normalize a tensor - I really need it to be normalized. But matmul/addition? Maybe there is wiggle room.
Big challenge would be whether any gains could compete with the economy of scale from NVidia.
I do think that a "better rng" can be interesting and useful in and of itself.
Thanks for the Normal Computing post, it felt more substantial.