[openquery]

> More likely (without careful vetting by the folks aggregating these models) is that the quality will go down as more and more AI-generated output gets subsumed into these models.

This assumes that there aren't algorithmic breakthroughs which reduce training/inference costs by several OOMs.

How much do these models need to do before people throw their hands in the air and say, ok this is happening. The Erdos unit distance problem, which as far as I understand was approached by multiple competent mathematicians was solved by a frontier model. Sure people argue there was no novelty there (I cannot comment as a non-mathematician) but it feels like they can draw lines laterally from deep knowledge in different fields (in this case combinatorics and algebraic number theory I believe) and solve problems.

Now if you have millions of instances running in parallel, all "probabilistic", working on frontier AI research I really don't see the blocker (and believe me I wish I did).

[taurath]

> How much do these models need to do before people throw their hands in the air and say, ok this is happening

What is "this"? Most people arguing against some of the more fervent predictions and promises of "inevitability" are people who are using these models in day to day - they see what the models can do, and what they struggle at.

> Now if you have millions of instances running in parallel, all "probabilistic", working on frontier AI research I really don't see the blocker (and believe me I wish I did).

My genuine prediction is that you'll get a lot of early results simply because you're applying attention to some low hanging fruit of problems, but then it will drop off due to the cost of tokens and the low rate of return. This doesn't mean that the models are especially capable of novel thought, just that we haven't algorithmically brute forced a problem with known solutions.

We would be seeing more success cases if the promises were true, setting aside AGI, human replacement, etc. We would see more, better products with more features that people would use. We wouldn't be having any arguments. The human replacement presupposes the models work in ways that they don't, and until proven otherwise, can't. I've watched those who embrace it fully flounder around on projects, some have lost their mind from the constant LLM validation, and I've seen companies go all in and then pull back based on both cost and efficacy over the last year.

I'm still waiting for the success case examples applied on a scale that would make any of the predictions come true.

[AnimalMuppet]

You're assuming that frontier AI research can "kick off the exponential" - that there exists such an exponential to kick off, and that frontier AI research can reach it soon, rather than in a couple centuries. Those are massive assumptions that have exactly zero empirical data behind them.

[mikgp]

> How much do these models need to do before people throw their hands in the air and say, ok this is happening.

It's quite interesting to me that your epistemology is basically "God in a box is inevitable" and not skepticism.

> Now if you have millions of instances running in parallel, all "probabilistic", working on frontier AI research I really don't see the blocker (and believe me I wish I did).

The blocker is lots of things, but for one? Compute. Millions of instances running parallel, all "probabilistic" testing different theories would require million of GPU clusters.

Two? For subjects like Physics - Atoms. You need to be able to run experiments. You need access to the real world.

Then there's the S curve. Algorithmic breakthroughs operate on orders of magnitude. Eventually we'll run out.

etc. et al.

[bigstrat2003]

> This assumes that there aren't algorithmic breakthroughs which reduce training/inference costs by several OOMs.

Yes, if one must assume something it is generally fair to assume that things will continue as they are. Research breakthroughs do happen, but they are not something for which you can predict the timing.

[danaris]

> This assumes that there aren't algorithmic breakthroughs which reduce training/inference costs by several OOMs.

Of course it does.

You cannot operate based on the assumption that you will hit a breakthrough in any given span of time. Breakthroughs are, by their very nature, unpredictable—both on when you will reach them, and on whether they exist to reach at all.

It's also assuming that there isn't a hardware breakthrough that will put us another several OOMs ahead on compute.

It's also assuming that there isn't a military "breakthrough" that leaves all our supply lines either bombed or hostile to us.

When you're forecasting, you can only operate based on what is known and knowable now.

Breakthroughs are unknowable. You cannot rely on them.