[ggm]

This is what I've always believed. Happy to be corrected or refuted!

You're looking at something which has unknowns to its shape. You do not have the luxury of an exhaustive test of all variances in the input, or model. You need an optimisation which explores the space, and allows you to intuit refinements to the model.

In some cases, you have the experimental data. So you are looking for a decision-logic over which part(s) to use, and how to interpret them.

You test with a random selection of inputs, models, conditions and see how they cluster. The questions about what to do, would be "how many" and "how random-y"

[sungx105]

I guess pared down to its absolute core, this is what Monte Carlo is - you just generate many a large ensemble of possible states.

But this simplified explanation misses out on one key aspect of Monte Carlo: sometimes different kinds of Monte Carlo moves can be designed that can allow it to more efficiently sample the phase space than other methods such as gradient descent.

Unfortunately, doing so is can be very involved, and is not always very general, so it isn't as easy to do as using other methods for exploring phase space.

[ggm]

Your answer feels like its saying "its only partly random, you use the outcome of your experiments to walk the space of possible solutions more carefully, in the light of experience" which would make it more like GLMM or something markov-y