[whinvik]

Almost every time a PDE is solved on a computer, it is a variational problem. Maybe neural networks are indeed good at this but I haven't seen any literature that shows that it is provably better. A reference would be good, especially to this point "But neural networks are very good parametric function approximators, generally better than what traditionally gets used in physics (b-splines or whatever)."

[currymj]

https://arxiv.org/abs/1909.08423 and https://arxiv.org/abs/1909.02487 are some examples I've been looking at recently.

[whinvik]

Thanks, not familiar with QM at all, but it seems to me from glancing through one of the papers that the neural network is used to replace a popular way of representing the wave function which itself is an Ansatz. Not very convincing, but of course, as I said not familiar with the background and so I may be overlooking something.

[currymj]

that's exactly it -- they take an existing form for the ansatz (or the general idea of it, at least), and make it more flexible by replacing pieces with neural networks that have many more parameters, while maintaining constraints required by physics. I think this will become very common in the future.

[whinvik]

That maybe true but what I was looking for is a more convincing way of showing that a neural network approximates a function better than other functions, such as say a b-spline. For example, you say that the neural network with many more parameters works better, but what if we had a b-spline with many more nodes.

[hnarayanan]

I don't know anything about anything but I'm willing to bet that the end result is very similar. They're "just" using neural networks as rich approximators.

[jhrmnn]

I'm an author of one of the arXiv papers above. One thing to consider is that the approximative power of a given parametric function is not the only criterium. Being able to optimize that function efficiently is as important. Neural networks excel in this. So the comparison you ask for most likely won't appear, because any other parametric ansatz with tens of thousands (or more) parameters would be impossible to optimize. At the least that's the case in quantum Monte Carlo, the domain of our paper. As for "provable", I also don't think that will appear. All the exact theorems about neural networks are way too abstract to be applicable to practical problems.

[btrettel]

> Almost every time a PDE is solved on a computer, it is a variational problem.

Not true. In computational fluid dynamics, variational methods are only one category out of many, and they aren't dominant.

[whinvik]

Its usually only finite difference methods that are not variational. But finite difference is dominant in academia, not in industry. And that is changing as well with methods such as the discontinuous Galerkin method. The more popular finite volume method in industry, can also be seen as a variational problem.

Yes, I exaggerated when I said that, but its still mostly variational problems.

[btrettel]

> The more popular finite volume method in industry, can also be seen as a variational problem.

By that standard, you could interpret almost any numerical method for PDEs used in academia or industry as variational (aside from some fringe ones). By "variational" I mean methods which are designed in a variational way from the start, not can be merely interpreted variationally.

[whinvik]

Well, it helps to see these connections. For example, realising that the lowest order DG method is finite volume lets one think about how to extend well studied finite volume properties to high order DG methods.