[mr_world]

Organic adaption and persistence of memory I would say are the two major advancements that need to happen.

Human neural networks are dynamic, they change and rearrange, grow and sever. An LLM is fixed and relies on context, if you give it the right answer it won't "learn" that is the correct answer unless it is fed back into the system and trained over months. What if it's only the right answer for a limited period of time?

To build an intelligent machine, it must be able train itself in real time and remember.

[specialist]

Yes and: and forget.

[8n4vidtmkvmk]

Why is forgetting important? Things can either have an end time where they are no longer applicable or things we thought were true turn out to be false but it's still useful to see where we went wrong.

I imagine humans are limited by the # of synapses we have so it's useful to forget but maybe machines can move the useless stuff to deep storage until it's dug out, in the same way certain things can trigger a deep memory in humans.

[lupire]

Which is cheaper, among these two logically equivalent things: reducing one weight, or increasing every other weight?

[specialist]

Do you remove dead code? Get rid of clutter? Ever try to change a habit?

[8n4vidtmkvmk]

Yeah, I delete dead code but it's important to remember why I wrote it that way in the first place and why I'm deleting it now. Doomed to repeat past mistakes and all that.