[fnordpiglet]

In my layman’s naive view the promise of generative AI is the ability to estimate highly dimensional non linear systems effectively and efficiently. At some level these can be viewed as solving non linear systems. In my career a specific class of important problems has been estimating systems of partial different equations, and specially stochastic partial differential equations. Monte Carlo methods are often the most computable estimations. I’ve often found we approach these things by either actually linearizing or estimating a linearization of the system and using linear algebra to solve, then transforming back. All these techniques requires enormous amount of computation and extremely complex math and numerical methods. To my naive understanding (I’m more of a core systems person that’s been adjacent to the work) quantum promises to help here by directly simulating the system.

Would this thermodynamic technique provide solutions to these sorts of non linear optimization and system solving problems? It feels from my reading it might, and in a simpler way to express.

Forgive my likely display of extraordinary ignorance.

[aifer4]

One way to think about these methods is that we are essentially implementing a Monte-Carlo algorithm physically, where on each "iteration" there is a matrix-vector multiplication. The physical system does this matrix-vector multiplication for us in constant time, so it does have an advantage over these digital methods. Not only that, but the "clock speed" of the physical system can be almost arbitrarily short, although this comes with an energy cost.

[fnordpiglet]

Yes that’s precisely what made me harken back to my solving of large stochastic PDE system questions :-)

The different though is instead of a matrix multiplication it’s a nonlinear optimization. The crucial part is the nonlinearity. But I assume given this technique is as you say Monte Carlo at its root, that shouldn’t specifically matter?

[aifer4]

You mentioned that such problems may be solved by solving a linear approximation of the non-linear problem (and I am in no way an expert in non-linear optimization). To the extent that the bottleneck in that approach is solving the resulting linear system, this method offers a speedup. We are also thinking about using similar thermodynamic methods to solve non-linear systems directly, but some of the nice properties of the harmonic oscillator are not present in that case, so it's currently not clear how much (if any) speedup is there.