Hacker News new | ask | show | jobs
by Enginerrrd 474 days ago
Strictly speaking I'm not sure if it does require learning if information representing the updated context is presented. Though it depends what you define as learning. ("You have tried this twice, and it's not working.") is often enough to get even current LLM's to try something else.

That said, your second paragraph is one of the best and most succinct ways of pointing out why current LLM's aren't yet close to AGI if though they sometimes feel like it's got the right idea.
1 comments
bashfulpup 474 days ago
In context learning, learning via training. Both are things we barely understand the mechanism of.

RAG is a basically a perfect example to understand the limits of in context learning and AI in general. It's faults are easier to understand but the same as any AI vs AGI problem.

I could go on but CL is a massive gap of our knowledge and likely the only thing missing to AGI.

link
tw1984 473 days ago
> RAG is a basically a perfect example to understand the limits of in context learning and AI in general.

How? RAG is not even in the field of AI.

link
bashfulpup 473 days ago
Long explanation. Simple terms, you can't use a fixed box to solve an unbounded problem space. If your problem fits within the box it works, if it doesn't, you need CL.

I tried to solve this via expanding the embedding/retrieval space but realized it's the same as CL and in my definition of it I was trying to solve AGI. I did a lot of unique algorithms and architectures but Unsuprisingly, I never solved this.

I am thankful I finally understood this quote.

"The first gulp from the glass of natural sciences will turn you into an atheist, but at the bottom of the glass God is waiting for you."

link