[reedwolf]

Bottom line:

"With the new database-based science, there is often no moment when the complex becomes simple enough for us to understand it. The model does not reduce to an equation that lets us then throw away the model. You have to run the simulation to see what emerges. For example, a computer model of the movement of people within a confined space who are fleeing from a threat--they are in a panic--shows that putting a column about one meter in front of an exit door, slightly to either side, actually increases the flow of people out the door. Why? There may be a theory or it may simply be an emergent property. We can climb the ladder of complexity from party games to humans with the single intent of getting outside of a burning building, to phenomena with many more people with much more diverse and changing motivations, such as markets. We can model these and perhaps know how they work without understanding them. They are so complex that only our artificial brains can manage the amount of data and the number of interactions involved."

[throwawaygh]

> The model does not reduce to an equation that lets us then throw away the model. You have to run the simulation to see what emerges.

This is true of simulation in general, not just data-drive models. E.g., a lot of applied mathematics uses PDE models that don't have closed-form solutions and so you just run a ton of simulations sweeping a parameter space.

> For example, a computer model of the movement of people within a confined space who are fleeing from a threat--they are in a panic--shows that putting a column about one meter in front of an exit door, slightly to either side, actually increases the flow of people out the door.

The crux of this type of science is that you don't know whether the computer simulations are telling you anything about reality. You just have to run real-world experiments and see what happens. And even if the experiment turns out to work, you still don't know for sure that your model was reasonable.

[mturmon]

It's also worth saying that there's an emerging discipline devoted to formalizing and developing tools for this kind of problem (complex computer models of real-world systems, say).

One decent place to start is this National Academies report [1] on Verification, Validation, and Uncertainty Quantification.

Verification = did you implement the math correctly in the computer;

Validation = does the implemented mathematical model compare against the real system in controlled experimentss

Uncertainty Quantification = analysis and prediction of the accuracy of the model approximation.

This work was given a big push by the nuclear test ban treaty - you have to really validate the model predictions in this case.

[1] https://www.nap.edu/catalog/13395/assessing-the-reliability-...

[newqer]

So you are suggesting we need to do a double blind test, by means of throwing a molotov cocktail at a few people gatherings?

[nxpnsv]

Double blind so that neither the thrower of recipients of said cocktail doesn't know wether it is real or placebo? It's the only way to know if they are panicking because of a bottle or a fireball...

[throwawaygh]

There are probably slightly more ethical ways of designing that study...