[aaimnr]

This naive belief in rationality on her part is troubling. It reminds me the simplicity of XVIII century enlightenment thinkers. No wonder that Nielsen, being accustomed with how DNNs work. represents the opposite view.

"Rationality" is at best a filter of rule constraints applied to small part of solutions that reach consciousness in any solution-seeking process. As we know eg. from Lakoff and Johnson ('metaphors we live by' etc.) most of reasoning is actually done using our sensory faculties, mostly spatial, which barely guarantee any kind of correctness.

If you try to be rational and cling to concepts while trying to figure out an answer to difficult problem, you obstruct the process rather than help it. Hence the popular advice to 'forget about the problem' so that the mind can figure it out by itself. How on earth is it rational? Rationality is only applied when checking a solution, like checking a proof, which - as we know - is orders of magnitude less computationally complex than finding a proof.

The mind is all about 'what works' not 'what's correct', rationality is pretty modern invention and to honestly think that we're primarily rational is a delusion. Tversky and Kahneman's work is another obvious counterexample.

[hammock]

This is why I can't stand lesswrong and the comments of lesswrongers here.

I believe you would love the book "The Romantic Economist" by Richard Bronk which outlines romantic ideals of creativity, entrepreneurship, and other magic as a necessary ingredient of progress that is missing from modern philosophy and science.

[aaimnr]

Well, it's really hard to tell which myth is causing more harm in our culture - myth about rationality or myths carried over from Romanticism. I'm not contrasting rationality with 'romantic' feelings (term that doesn't explain a lot), but rather conceptual with pre-conceptual.

[kruhft]

Excellent comment. Thank you.