Hacker News new | ask | show | jobs
by Inufu 260 days ago
Well, growth has been on this exponential already for 5+ years (for the METR eval), and we are at the point where models are very close to matching human expert capabilities in many domains - only one or two more years of growth would put us well beyond that point.

Personally I think we'll see way more growth than that, but to see profound impacts on our economy you only need to believe the much more conservative assumption of a little extra growth along the same trend.
3 comments
ModernMech 260 days ago
> we are at the point where models are very close to matching human expert capabilities in many domains

This is not true because experts in these domains don't make the same routine errors LLMs do. You may point to broad benchmarks to prove your point, but actual experts in the benchmarked fields can point to numerous examples of purportedly "expert" LLMs making things up in a way no expert would ever.

Expertise is supposed to mean something -- it's supposed to describe both a level of competency and trustworthiness. Until they can be trusted, calling LLMs experts in anything degrades the meaning of expertise.

link
bawolff 260 days ago
> we are at the point where models are very close to matching human expert capabilities in many domains

That's a bold claim. I don't think it matches most people's experiences.

If that was really true people wouldn't be talking about exponential growth. You don't need exponential growth if you are already almost at your destination.

link
hnlmorg 260 days ago
Which domains?

What I’ve seen is that LLMs are very good at simulating an extremely well read junior.

Models know all the tricks but not when to use them.

And because of that, you’re continually have to hand hold them.

Working with an LLM is really closer to pair programming than it is handing a piece of work to an expert.

The stuff I’ve seen in computer vision is far more impressive in terms of putting people out of a job. But even there, it’s still highly specific models left to churn away at tasks that are ostensibly just long and laborious tasks. Which so much of VFX is.

link