[jdlshore]

> As soon as we can prompt…

This is the fundamental error I see people making. LLMs can’t operate independently today, not on substantive problems. A lot of people are assuming that they will some day be able to, but the fact is that, today, they cannot.

The AI bubble has been driven by people seeing the beginning of an S-curve and combining it with their science-fiction fantasies about what AI is capable of. Maybe they’re right, but I’m skeptical, and I think the capabilities we see today are close to as good as LLMs are going to get. And today, it’s not good enough.

[Workaccount2]

Getting gold in the math Olympiad is a pretty strong indicator of operating independently on substantive problems.

A year ago they need an extensive harness to get silver, and two years ago they could hardly multiply 1000x10000.

Terence Tao tweeted yesterday about using GPT5 to help quickly solve a problem he was working on.

[saberience]

Yes but why did ChatGPT work on math Olympiad problems? Because it got a prompt giving it the instruction and context etc.

Why did GPT5 help Terence Tao solve a math problem, because he gave it a prompt and the context etc.

None of these models are useful without a human prompting them and giving it tasks, goals, context etc, they don't operate independently, they don't get ideas of work to be done, they don't operate over long time horizons, they can't accept long term goals and sub-divide those goals into sub goals, and sub tasks etc.

They are useless without humans telling them what to do.

[esafak]

Why don't you stick them in a robot, give them agency, continuously train them, and see what happens? Be careful what you ask for.

[Workaccount2]

You should see what happens when you let them talk to each other

[WA]

Errors compound? Context drift?

[Workaccount2]

Try it, and let them pick the topic. Though they will probably pick AI development, mysteriously it seems to be their favorite topic...