[samgriesemer]

You might take a look at The Bitter Lesson [1], it's referenced by the article and linked around on this thread.

> One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.

> The second general point to be learned from the bitter lesson is that the actual contents of minds are tremendously, irredeemably complex; we should stop trying to find simple ways to think about the contents of minds, such as simple ways to think about space, objects, multiple agents, or symmetries.

[1]: http://www.incompleteideas.net/IncIdeas/BitterLesson.html

[aabhay]

I did take a look at the article before writing my comment, and I disagree with its premise as well as conclusion. The human mind has a variety of specialized functions. Our ocular cortex does a sort of convolution on a 2D field with three color and one alpha ‘sensors’ (cones and rods). If specialization were less powerful than generality, why isn’t our brain one giant lobe with no diversity in neuron topography?

The idea that specialization is not as powerful as computation fails the most basic test of a proactive, rather than retroactive, theory. Can you make proactive claims about what works in any given domain? Is the solution to take the hungriest algorithm and apply it? What about feature engineering, cleaning, parameter tuning, analysis, etc.? Is the most power hungry solution still the most effective? In my opinion, part of the reason humans aren’t just giant computation blobs is that we thrive on constraints (physical, sexual, emotional).