[xzephyr]

With uncertainty you just change your objective function to an evaluation function of any possible scenario scaled with its probability. It doesn't change how optimization works. Again this formulation is only easy to solve in very limited context. For example in economics people have been working with the oversimplified supply-demand curves precisely because they are usually the dominating factors and in practice a sufficiently accurate model works just fine. This model only gives insights for why different ways of reasoning works, not actually providing the panacea.

The good thing about gradient descent is that you do NOT need to have a model, you just need to focus on a few parameters and figure out what is the direction for best improvement from a current relatively good point, where the other billions of parameters are already accounted for and assumed independent from the direction you are going.

[jorleif]

It seems you are assuming there are direct observations. Musk is talking about generating hypothetical observations from a model (the cost of the battery is bounded from below by the cost of the battery materials). This sort of bounding does not always work outside physics, because the uncertainties are so ill behaving.