Hacker News new | ask | show | jobs
by bashfulpup 469 days ago
In context learning, learning via training. Both are things we barely understand the mechanism of.

RAG is a basically a perfect example to understand the limits of in context learning and AI in general. It's faults are easier to understand but the same as any AI vs AGI problem.

I could go on but CL is a massive gap of our knowledge and likely the only thing missing to AGI.
1 comments
tw1984 469 days ago
> RAG is a basically a perfect example to understand the limits of in context learning and AI in general.

How? RAG is not even in the field of AI.

link
bashfulpup 469 days ago
Long explanation. Simple terms, you can't use a fixed box to solve an unbounded problem space. If your problem fits within the box it works, if it doesn't, you need CL.

I tried to solve this via expanding the embedding/retrieval space but realized it's the same as CL and in my definition of it I was trying to solve AGI. I did a lot of unique algorithms and architectures but Unsuprisingly, I never solved this.

I am thankful I finally understood this quote.

"The first gulp from the glass of natural sciences will turn you into an atheist, but at the bottom of the glass God is waiting for you."

link