Hacker News new | ask | show | jobs
by activatedgeek 1917 days ago
And, guess what? Since, the Galerkin approximation requires one to choose a basis that is appropriate to the problem at hand, we now have a deep learning solution too (since neural network learning is essentially equivalent to learning an adaptive basis).

It is called the Deep Galerkin Method [1]. In a nutshell, the method directly minimizes the L2 error over the PDE, boundary conditions and initial conditions. The integral is tricky though, and computed via a Monte Carlo approximation.

[1]: https://arxiv.org/abs/1708.07469
1 comments
pmiller2 1917 days ago
Why would you use the Monte Carlo method when the quasi-Monte Carlo method converges so much more quickly? I admit, I am a little biased, because I worked on some QMC stuff in grad school, but it works really, really well in practice.

https://en.wikipedia.org/wiki/Quasi-Monte_Carlo_method

link
contravariant 1916 days ago
If the number of dimensions is high enough then pretty much all points are far away from each other so there's no real need to worry about points clustering. You're at a much greater risk of simply missing interesting parts of your space than you are of oversampling any part of it.
link
blake1 1916 days ago
This can be especially bad if one corner hides a catastrophe. Any automatic method is likely to have an exponential struggle to sample appropriately. At a certain point, you just need to apply your own understanding of the problem you are trying to solve, and resort to an ad hoc method to do the “right” thing.
link
pmiller2 1916 days ago
That's exactly the problem QMC methods solve. By choosing a sequence of sample points of low discrepancy, you make sure to sample the entire space as evenly as possible.
link
contravariant 1915 days ago
It does but if you've got 100s of dimensions then you have no hope of ever sampling anything close to the entire space. Even if you just pick 2 points per dimension you've got no way of trying every combination.
link
pvitz 1916 days ago
Just a guess, but if you think about Sobol sequences, maybe the available dimensions get exhausted in this use case and then QMC doesn't perform well anymore.
link