[spot5010]

The human analogy would be that I don’t remember everything in the books I have read, but I do recall reading a particular book and can always look it up.

So, is there a way to train a neural network and then tune it forget a lot of the facts that can be easily retrieved, but keep the intelligence.

[Gigachad]

LLMs can do that. The one that's built in to google search is relatively stupid, but it just uses web pages to fill in the gaps.

The downside is it is incredibly easy to be tricked by a single false bit of info.

[mandeepj]

> So, is there a way to train a neural network and then tune it forget a lot of the facts that can be easily retrieved, but keep the intelligence.

MoE (mixture of experts)?

[sdlion]

I guess that's why is an active research area, and an interdisciplinary one, what is intelligence? Is what have you practiced a ton of times? Is what you have purposely and efficiently practice? What is the implication of "purposely" doing something and how much memory is involved into it? If memory is involved how much of it is relevant? What is problem solving or wisdom? creativity? How much diversity in your memory do you need for creativity?

So what should you cull and how much? There's already techniques in CNNs to trim unused or less active neuron paths to reduce a model's size, but how do you (and how much) do it in a general LLM? A product that will be used without supervision from a child to an 80yo elder?

[fhn]

I read a lot in middle school and can't recall any of them. What books did you read in college? What books did you read in high school?Maybe you have photographic memory.