[nicholast]

The root enabling factor was not only the reinforcement learning, it was the simulation. Put differently, any environment that you can simulate within sufficient accuracy, you can derive robust and efficient control by reinforcement learning.

[avs733]

I'm probably being excessively pedantic but I would suggest sufficient fidelity is a more appropriate phrase than sufficient accuracy.

The exact precision doesn't seem to be the underlying problem being solve, what matters is being able to craft a stronger understanding of relationships of underlying phenomena that can then be leveraged to product new control theories.

[adrianmonk]

I think they are just talking about how accurately the model reflects the real world.

If someone draws a map and it has some streets with the wrong names, you say, "This map is not accurate." It doesn't mean you are talking about something numerical.

[agoose77]

Right - in scientific parlance, accuracy and precision have distinct meanings: https://www.antarcticglaciers.org/wp-content/uploads/2013/11...

In this case accuracy is appropriate :)

[avs733]

I would argue it isn't. I love that graphic (I use it in classes frequently) but it's limiting here.

Accuracy is about how close to the underlying behavior that is being modeled is. In this case, the goal is modeling the controllers they are using.

Precision is about how stable I can make my controller parameters and how stable they can make the behavior or the underlying fusion.

What they did was changed the fidelity of the model. Rather than parameterizing it using a known controller model (e.g., a PID controller) they used the neural network to fundamentally discover the important parameters that they can understand and how they interact.

The simulation can be accurate or not accurate entirely independently of how much fidelity of the actual phenomenon you model. You need to have a high fidelity model, and that model needs to be accurate, and they behavior needs to be precise and stable.

An example would be how wing profiles in aircraft are simulated/modeled. I can do it in 2D with high accuracy and high precision - it's a GOOD 2D simulation. But its low fidelity. I might be better off with a lower accuracy model of a high fidelity model, like a 3D model that lets me look at how the ends of the wingtip perform as well. Increasing the accuracy of that model might make the results less stable (because numerical techniques...great). A lot of improvements in computing power and simulation techniques have resulted in the ability to simulate systems more completely and learn about the interactions - not just making the simulation more accurate to what it contains.

[chairmanwow1]

I think this was an important distinction to make~

[elcomet]

That's not true of any environment though. You still need some characteristics like a not-too-sparse reward, and certainly many other things.

[amelius]

You can do it without simulation, but you'd have to build a million tokamaks.

[willis936]

Or one TCV.

[Sosh101]

Is this really true with complex chaotic systems?

[PartiallyTyped]

If the reward function is sufficiently dense, then likely yes. RL uses function approximators to solve problems, so in theory the perturbations of the chaotic system shouldn't matter as much.

[EGreg]

How to simulate human humor responses? Then we can evolve AIs that are funnier and more resourceful than anyone on the planet.

How to simulate human connection, attraction and arousal responses? Then we could have bots that are smoother talking and far more preferable to human lovers.

How to simulate human responses to nutrition? Then we could have individualized recommendations for diets, achieving goals like holistic health or muscle definition and addressing issues like diabetes.

[ksubedi]

These are all complex systems where figuring out how to simulate them is a much difficult task than even reinventing reinforcement learning from scratch.