[saberience]

Yes but why did ChatGPT work on math Olympiad problems? Because it got a prompt giving it the instruction and context etc.

Why did GPT5 help Terence Tao solve a math problem, because he gave it a prompt and the context etc.

None of these models are useful without a human prompting them and giving it tasks, goals, context etc, they don't operate independently, they don't get ideas of work to be done, they don't operate over long time horizons, they can't accept long term goals and sub-divide those goals into sub goals, and sub tasks etc.

They are useless without humans telling them what to do.

[esafak]

Why don't you stick them in a robot, give them agency, continuously train them, and see what happens? Be careful what you ask for.

[Workaccount2]

You should see what happens when you let them talk to each other

[WA]

Errors compound? Context drift?

[Workaccount2]

Try it, and let them pick the topic. Though they will probably pick AI development, mysteriously it seems to be their favorite topic...