[keenmaster]

Key quote from the conclusion: "Neural Networks are not just good for things we don't know how to solve, they can provide massive performance gains on problems we already know how to solve."

That quote is in reference to tasks such as physics simulation. There is an incredible GIF in the OP which shows a digital mannequin being manipulated, with its dress flowing in a hyper-realistic manner due to ML physics simulation. It would be uncanny to see that type of simulation combined with AR.

I'm curious to what extent ML physics simulation may be beneficial for self-driving cars. Generally, we as drivers know the physical properties of objects that we can collide with. Cars don't have that understanding, so they might "think" that colliding with a large paper bag is unacceptable. Stopping suddenly because of that paper bag may be fatal.

[ThePhysicist]

I don’t think the cloth simulation is based on ML, this looks like a normal physics-based solver. If I understand the article correctly the author just uses this as an example of complex behavior that could be learned using ML, but he/she doesn’t indicate that this was actually done using ML.

I know some papers that try to improve physics simulations with deep learning and I think it’s definitely possible, not sure though if it can really improve most physics-based simulations.

Physical models are already highly condensed and have the advantage of being interpretable, deep learning has a long way to go before it could be used as a replacement, IMO.

[keenmaster]

A “300-5000x” gain in simulation speed for low-complexity simulations would be impressive. If a blade of grass is low complexity, then I assume that would free up resources from basic environmental rendering for more complex objects, and greater complexity in those objects (which themselves might benefit from ML, but with <300x gains). Until everyone has near infinite computing power, that’s a de facto improvement in simulation quality right?

[ThePhysicist]

How would you reach such a gain though? Most physics-based simulations are really efficient already. For example, cloth simulations usually work using a grid of points (finite element approach) that are evolved using a differential equation and a constraint solver (not an expert in this particular area but I wrote e.g. electrodynamic simulations). Each point is usually only connected to its immediate neighbors, so the update algorithm only needs to do some simple arithmetic operations for each point. Often there's another step afterwards that ensures the constraints are met. All this can be executed really efficiently on a GPU already. Here's an example of an electrodynamics simulation I wrote: https://github.com/adewes/fdtd-ml

To achieve a 300-500 speedup of this seems impossible because for a single grid point we only do a few numerical operations to update it, so it's hard to see how one could reduce that much further as even an ML-based model will need to update each grid point to maintain the level of detail.

I think there are definitely other areas where ML can speed up things, but IMO cloth simulation is just a really bad example because it's a problem that can be solved using a nearest-neighbors approach with rather simple equations. Problems where you have non-local interactions or more complex dynamics might profit more from ML, but most physics problems can be solved faster with much simpler approaches, I think.

[petermcneeley]

They used low inertia cloth and solids because these are the kinds of scenarios that this technique can best approximate.

Once you buy into the concept that your simulation can be represented by a very limited number of parameters it's not hard to suggest that this small state vector can be mutated from state to state using ML.

The issue with this technique is PCA not ML.

I can certainly see this technique being used in place of some existing simulation or animation based secondary motion in video games.

Note that adding up the contributions of 256 basis vectors might be more expensive than a per vertex cloth simulation.

I am experienced in these areas and have done grad computational physics and work on game engines for a living.