[xzephyr]

The so-called reason by analogy is just gradient descent to approach local optima in optmization. The "by first principle" is just providing an approximate (ideally convex) model and solve it analytically for global optima. The problem is generally very hard (#P hard), both for formulating the problem and solving it. Global optimum of course in theory is better, but the quality of your objective function and constraints could easily offset this advantage. If you can come up with a simple linear programming problem for the battery example - that's great, but most likely you won't due to the prescence of competitor, market and policy constraint.

[jorleif]

I agree, but would like to add the aspect of uncertainty. When reasoning from analogy, you usually have good statistical knowledge about the properties of the problem. For example, the targetted product category may exist, and customer behavior is known, and therefore predicting what would happen in some nearby configuration is usually somewhat accurate. On the other hand, in the first principles case you need to have a very accurate theory, because you are "far away" from what exists currently. If the theory in questions concerns physics, as for Musk, then this can work well. On the other hand if it is about social sciences and you need to predict the behaviors of customer from some kind of first-principle model (e.g. rational agent models), then you are very likely to make mispredictions, since human behavior is complicated, and the theories are inaccurate and overly general.

[xzephyr]

With uncertainty you just change your objective function to an evaluation function of any possible scenario scaled with its probability. It doesn't change how optimization works. Again this formulation is only easy to solve in very limited context. For example in economics people have been working with the oversimplified supply-demand curves precisely because they are usually the dominating factors and in practice a sufficiently accurate model works just fine. This model only gives insights for why different ways of reasoning works, not actually providing the panacea.

The good thing about gradient descent is that you do NOT need to have a model, you just need to focus on a few parameters and figure out what is the direction for best improvement from a current relatively good point, where the other billions of parameters are already accounted for and assumed independent from the direction you are going.

[jorleif]

It seems you are assuming there are direct observations. Musk is talking about generating hypothetical observations from a model (the cost of the battery is bounded from below by the cost of the battery materials). This sort of bounding does not always work outside physics, because the uncertainties are so ill behaving.

[possibilistic]

I love this explanation. Do you happen to blog about optimization topics?

[xzephyr]

Ah, sorry no. I'm just interested in AI in general and constrained optimization has been the what I found to be the most general model for solving and giving insights into AI problems or any problems that require "intelligence".

[NamTaf]

If you're asking waht I think you're asking, then what you're then looking for is called 'numerical analysis'. Specifically, the grandparent was describing this [1]. Here is an OCW link for a good primer in to various introductory numerical analysis processes for Engineering [2].

[1]: https://en.wikipedia.org/wiki/Gradient_descent

[2]: http://ocw.mit.edu/courses/mechanical-engineering/2-993j-int...

[xzephyr]

Numerical Analysis is too broad for this topic. The precise category should be:

https://en.wikipedia.org/wiki/Mathematical_optimization

The hierarchy of topics in this area: http://en.wikipedia.org/wiki/Category:Mathematical_optimizat...

This is also why Google is paying millions of dollars for a quantum computer. Being able to solve complex optimization problems efficiently almost partially translates to access to higher intelligence.

http://googleresearch.blogspot.com/2013/05/launching-quantum...

[jrsnyder]

Seconded.