[jannyfer]

Great summary.

I’ve been reading a pop neuroscience book called Incognito (2011).

In it, the author talks about how the brain is a group of competing sub-brains of many forms, and the brain might have several ways of doing the same thing (e.g. recognizing an object). The author also posited that the lack of AI progress back then was due to the fact that there are no constantly competing sub-brains. Our brains are always adjusting and trying new scenarios.

I was struck by how similar these brain observations were to recent developments in AI and LLMs.

The book is full of cool stories, even if some of them are now recognized as non-reproducible. I recommend!

[go_elmo]

In the end - an AI should have these competing subsystems in one system - just as our brains are one system. What I find extremely interesting is how perception and thinking differs from person to person too - it was a "taboo" topic to call this neurodiversity - just as other genetic traits, but AI makes this relevant more than ever imo. Sure, its complicated and much comes from nurture (Nurture vs nature.. as exposure/epigeneticd vs genetics) but there sure are markable differences - the ones starting to stand out are e.g. adhd / autistic people, but Im sure it wont stay just there over time!

[samwillis]

You touch in an important topic here, how our understanding of AI/ML/LLMs will influence our "understanding" of the human brain and intelligence.

My fear is that we will ascribe too much human behaviour to that we see in and understand of our AI inventions, and that this will result in the dehumanisation of people.

So essentially my fear is what we justify doing to each other due to AI, rather than what "AGI" could do to us.

[lacrimacida]

I started seeing this dehumanisation spring up even here on HN, comparing LLMs with human brains and human thoughts as similar regurgitation to LLMs. I’m afraid it will get worse as this technology advances

[simonh]

Even within my immediate family we seem to have distinct differences in our conscious experience. My wife has very little visual or auditory experience of thought, no inner voice even when reading a book. While I mostly experience speaking as a continuous stream of words coming basically from my subconscious, with only a vague sense of what's coming up, one of my daughters says she is consciously aware of the exact words she is going to say several seconds in advance. It's like she has the ability to introspect her internal speech buffer, while I can't.

So while I'm sure there are a lot of custom tuned, problem specific hardware structures in our brain architecture, we do seem to learn how to actually use that hardware individually. As a result we seem to come up with a diverse range of different high level approaches.

[esafak]

> The author also posited that the lack of AI progress back then was due to the fact that there are no constantly competing sub-brains.

That became popular in neural networks after the introduction of dropout regularization, which forced neurons to "co-adapt" and learn to do each others' jobs. Large, over-specified models also provide a natural setting for co-adaptation.

[moffkalast]

Isn't dropout just there to avoid overfitting? This is more like a mixture of experts type architecture.

[esafak]

That is one lens to view it through. Co-adaptation reduction is another, and it is an intuitive one: generalization ability is improved if a neuron has to support multiple contexts instead of relying on other neurons to lift the weight, if you pardon the pun.

Improving neural networks by preventing co-adaptation of feature detectors https://arxiv.org/abs/1207.0580

[masswerk]

In fact, this is what psychoanalysis and the notion of the unconscious (as opposed to "subconscious processes") was all about. (And it's also, where the "talking cure" found its leverage.)

[wilhelm____]

thank you!

[throwuwu]

How are they similar?