The idea that progress is “slow” in the AI space is absurd. These are some of the fastest growing products and companies of all time. The models are still improving a surprising amount.
It's not that absolute progress is slow, it's extremely slow compared to the predictions. It might be fast in absolute terms, but the "50% of coders will be obsolete by 2023" has been renewed every six months, and it's becoming increasingly clear that there's a real chance it might not ever happen.
„Coders being obsolete” is not a measure of AI capabilities. I see coders being more busy than ever before. I see people without coding knowledge getting more behind. The gap is widening, not shrinking.
I see people without coding ability catching up to learned coders. AI is a huge force multiplier for people who don't have hands on, detailed technical knowledge, AI can increasingly handle that and just needs a human to steer it more broadly.
It's huge force multiplier for people who have hands on, detailed technical knowledge as well.
3 * N < 42 * N
42 - 3 < N * (42 - 3)
It helps to know layer you're working on.
People seem to make mistake of thinking how good LLMs are around tasks that they are familiar with and extrapolating it to whole population.
It's good mental exercise to think about how little you can do compared to expert on tasks you never thought of working.
Ie. if you're programmer or know something about finance, don't think how much it enables you to do better coding or investing, think instead how much it doesn't enable you to work on something you don't know like maybe molecular biology or visual special effects – it's all there but it's much better multiplier for people who do know their shit.
Knowing layer you're working on helps a lot, it gets multiplied.
Knowing programming is becoming more fundamental skill than ever before as it lies at the foundation of almost everything else.
I suspected you felt that way even though it hasn’t been my personal experience.
I’ve heard people say older models can’t do X, when I used that way etc. I suspect people are applying their own learning curve as part of their assessment of progress, you get better at writing prompts and it feels like the model improved.
Which is why I’m saying we need some objective metrics to judge predictions of actual capacity.
I mean you wanted something objective, and they are. I don’t know why you’re being dismissive of them, they’re a huge element of what drives model development forward.
These companies aren’t just making stuff up, they really do want to improve the models, and the models really are improving.