[sebastiennight]

Compared to someone who graduated middle school on November 30th, 2022 (2.5 years ago, would you say that today's gemini 2.5 pro has NOT gained intelligence faster?

I mean, if you're a CEO or middle manager and you have the choice of hiring this middle schooler for general office work, or today's gemini-2.5-pro, are you 100% saying the ex-middle-schooler is definitely going to give you best bang for your buck?

Assuming you can either pay them $100k a year, or spend the $100k on gemini inference.

[Jensson]

> would you say that today's gemini 2.5 pro has NOT gained intelligence faster?

Gemini 2.5 pro the model has not gained any intelligence since it is a static model.

New models are not the models learning, it is humans creating new models. The models trained has access to all the same material and knowledge a middle schooler has as they go on to learn how to do a job, yet they fail to learn the job while the kid succeeds.

[ben_w]

> Gemini 2.5 pro the model has not gained any intelligence since it is a static model.

Surely that's an irrelevant distinction, from the point of view of a hiring manager?

If a kid takes ten years from middle school to being worth hiring, then the question is "what new AI do you expect will exist in 10 years?"

How the model comes to be, doesn't matter. Is it a fine tune on more training data from your company docs and/or an extra decade of the internet? A different architecture? A different lab in a different country?

Doesn't matter.

Doesn't matter for the same reason you didn't hire the kid immediately out of middle school, and hired someone else who had already had another decade to learn more in the meantime.

Doesn't matter for the same reason that different flesh humans aren't perfectly substitutable.

You pay to solve a problem, not to specifically have a human solve it. Today, not in ten years when today's middle schooler graduates from university.

And that's even though I agree that AI today doesn't learn effectively from as few examples as humans need.

[Jensson]

> Surely that's an irrelevant distinction, from the point of view of a hiring manager?

Stop moving the goalposts closer, that you think humans might make an AGI in the future doesn't mean the current AI is an AGI just because it uses the same interface.

[ben_w]

Your own comment was a movement of the goalposts.

Preceding quotation to which you objected:

> A middle schooler has general intelligence (they know about and can do a lot of things across a lot of different areas) but they likely can't replace white collar workers either.

Your response:

> Middle schoolers replace white collars workers all the time, it takes 10 years for them to do it but they can do it.

So I could rephrase your own words here as "Stop moving the goalposts closer, that you think a middle schooler might become a General Intelligence in the future doesn't mean the current middle schooler is a General Intelligence just because they use the same name".

[Jensson]

Its the same middle schooler, nobody gave a time limit for how long it takes the middle schooler to solve the problem. These AI models wont solve it no matter how much time spent, you have to make new models, like making new kids.

Put one of these models in a classroom with middle schoolers, and make it go through all the same experiences, they still wont replace a white collar worker.

> a middle schooler might become a General Intelligence in the future

Being able to learn anything a human can means you are a general intelligence now. Having a skill is narrow intelligence, being able to learn is what we mean with general intelligence. No current model has demonstrated the ability to learn arbitrary white collar jobs, so no current model has done what it takes to be considered a general intelligence. The biological model homo sapiens have demonstrated that ability, thus we call homo sapiens generally intelligent.

[ben_w]

> Its the same middle schooler, nobody gave a time limit for how long it takes the middle schooler to solve the problem.

Yeah they do. If a middle schooler take 40 hours to solve a maths exam, they fail the exam.

> These AI models wont solve it no matter how much time spent, you have to make new models, like making new kids.

First: doesn't matter, "white collar jobs" aren't companies aren't paying for seat warmers, they're paying for problems solved, and not the kinds of problems 11 year olds can do.

Second: So far as I can tell, every written exam that not only 11 year olds but even 16 year olds take, and in many cases 21 year olds take, LLMs ace — the problem is coming up with new tests that describe the stuff we want that models can't do which humans can. This means that while I even agree these models have gaps, I can't actually describe those gaps in a systematic way, they just "vibe" like my own experience of continuing to misunderstand German as a Brit living in Berlin.

Third: going from 11 years old to adulthood, most or all atoms in your body will be replaced, and your brain architecture changes significantly. IIRC something like half of synapses get pruned by puberty.

Fourth: Taking a snapshot of a model and saying that snapshot can't learn, is like taking a sufficiently detailed MRI scan of a human brain and saying the same thing about the human you've imaged — training cut-offs are kinda arbitrary.

> No current model has demonstrated the ability to learn arbitrary white collar jobs, so no current model has done what it takes to be considered a general intelligence.

Both "intelligence" and "generality" are continuums, not booleans. It's famously hard for humans to learn new languages as they get older, for example.

All AI (not just LLMs) need a lot more experience than me, which means my intelligence is higher. When sufficient traing data exists, that doesn't matter because the AI can just make up for being stupid by being stupid really fast — which is how they can read and write in more languages than I know the names of.

On the other hand, LLMs so far have demonstrated — at the junior level of a fresh graduate of 21, let alone an 11 year old — demonstrated algebra, physics, chemistry, literature, coding, a hundred or so languages, medicine, law, politics, marketing, economics, and customer support. That's pretty general. Even if "fresh graduate" isn't a high standard for employment.

It took reading a significant fraction of the internet to get to that level because of their inefficiency, but they're superhumanly general, "Jack of all trades, master of none".

Well, superhuman compared to any individual. LLM generality only seems mediocre when compared to the entire human species at once, these models vastly exceed any single human because no single human speaks as many languages as these things let alone all the other stuff.

[sebastiennight]

This argument needlessly anthropomorphizes the models. They are not humans nor living entities, they are systems.

So, fine, the gemini-2.5-pro model hasn't gotten more intelligent. What about the "Google AI Studio API" as a system? Or the "OpenAI chat completions API" as a system?

This system has definitely gotten vastly smarter based on the input it's gotten. Would you now concede, that if we look at the API-level (which, by the way, is the way you as the employer do interact with it) then this entity has gotten smarter way faster than the middle-schooler in the last 2.5 years?

[Jensson]

But its the AI researchers that made it smarter, it isn't a self contained system like a child. If you fired the people maintaining it and it just interacted with people it would stop improving.

[ben_w]

The brain of a child is not self-contained either. Neither is the entire complete child themselves — "It takes a village to raise a child", to quote the saying.

The entire reason we have a mandatory education system that doesn't stop with middle school (for me, middle school ended age 11), is that it's a way to improve kids.

[sebastiennight]

1. The child didn't learn algebra on its own either. Aside from Blaise Pascal, most children learned those skills by having experienced humans teach them.

2. How likely is it that we're going to fire everyone maintaining those models in the next 7.5 years?

[Jensson]

> The child didn't learn algebra on its own either. Aside from Blaise Pascal, most children learned those skills by having experienced humans teach them.

That is them interacting with an environment. We don't go and rewire their brain to make them learn math.

If you made an AI that we can put in a classroom and it learns everything needed to do any white collar job that way then it is an AGI. Of course just like a human different jobs would mean it needs different classes, but just like a human you can still make them learn anything.

> How likely is it that we're going to fire everyone maintaining those models in the next 7.5 years?

If you stop making new models? Zero chance the model will replace such high skill jobs. If not? Then that has nothing to do with whether current models are general intelligences.

[int_19h]

Your brain does rewire itself as you learn.

Here's a question for you. If we take a model with open weights - say, LLaMA or Qwen - and give it access to learning materials as well as tools to perform training runs on its weights and dynamically reload those updated weights - would that constitute learning, to you? If not, then why not?

[ac29]

> Gemini 2.5 pro the model has not gained any intelligence since it is a static model.

Aren't all the people interacting with it on aistudio helping the next Gemini model learn though?

Sure, the results of that wont be available until the next model is released, but it seems to me that human interaction/feedback is actually a vital part of LLM training.

[Jensson]

It wont get smart enough without the researchers making architectural updates though, current architecture wont learn to become a white collar worker just from user feedback.