Hacker News new | ask | show | jobs
by kybernetikos 35 days ago
Current LLM architecture doesn't learn - and you're right this is a huge piece that normal folks fail to understand, since in many ways, it's the opposite of what years of AI research has been trying to create.

However, I think it's important to remember that LLMs are embedded in larger systems, and those larger systems do learn.
2 comments
baq 35 days ago
If I was a frontier lab and I solved continual learning, as of today I would absolutely not release it - the society isn't ready for this; society isn't even ready for widespread diffusion of current publicly available frontier models.

If however I was a frontier lab who solved continual learning and my competitor also solved and released it, I would release mine immediately, obviously.

The point is, continual learning might be solved already, we just don't know and those who might know would rather keep their mouths shut. It isn't my base case (financial situation of frontier labs is such that they'd probably release immediately as long as they have inference compute to serve this revolutionary capability), but it isn't impossible.

link
bigyabai 35 days ago
You're not a frontier lab, the shareholders own those. And if shareholders get a private briefing about an unprecedented breakthrough in continual learning, they would announce it from the rooftops to take credit for the progress ASAP and reap the rewards for their stock value.

The only lab that I can exempt from this is DARPA.

link
baq 35 days ago
Shareholders are not insiders. Public companies do secret projects all the time of which shareholders know absolutely nothing about and may never learn the details of them if they get cancelled.
link
2wdfsd 35 days ago
Private market dynamics are not the same buddy.
link
baq 34 days ago
Everyone owns them at this point and Google is outright public.
link
wrer 34 days ago
No youre missing the point of the poster - disclosures in private markets are different than public, especially in the context of large commitments - the company has no choice.
link
lukewarm707 35 days ago
exactly like you said - the harness might learn.

we do also have training on synthetic data. it might compound.

link