[sudb]

> By reaching into the voids of its embedding space and returning tokens related to nonexistent semantics. Or, if you like, "hallucinating". The hallucinations which are useful we might call "novel".

This sounds to me like an admission that LLMs are not just doing a stochastic search, then.

> close enough facsimile to intelligence

What's the distinguishing criteria then? How can you tell the difference?

[27183]

I think we must be talking past eachother. I define stochastic search as a search process with randomness injected into it that can return the following things:

- Something contained in the data set, not necessarily the same thing for every iteration of a given query

- Something not contained the data set (hallucination), not necessarily the same thing for every iteration of a given query

Does that clear it up?

> What's the distinguishing criteria then? How can you tell the difference?

All the ways they fail to exhibit intelligence. They can't learn. They can't adapt. They can't reason abstractly. They can't count. Etc...

[sudb]

Ah it seems you are a stochastic parrot believer (https://en.wikipedia.org/wiki/Stochastic_parrot). What are your responses to the Expert Rebuttals section?

I find the rebuttals pretty convincing - that there seems to be some emergent behaviour that is not simply just next-token-prediction, or that the ability to do accurate next-token-prediction requires something "extra" that LLMs have.

> All the ways they fail to exhibit intelligence

Another implicit admission that there _are_ ways that LLMs exhibit intelligence?

[27183]

> there seems to be some emergent behaviour that is not simply just next-token-prediction, or that the ability to do accurate next-token-prediction requires something "extra" that LLMs have.

The next step then would be to design and conduct experiments that isolate this effect. Figure out how to make it happen reliably and in such a way that you know it's actually happening as opposed to just something you're imagining. Isolate it or distill it so it can be studied directly. Until then, it's easiest to dismiss it as imaginary.

[sudb]

> Figure out how to make it happen reliably

And you're happy that the replication of LLMs across many foundation model companies is insufficiently reliable?

> just something you're imagining

So the alternative explanation you're suggesting to emergent LLM behaviour is mass independently-corroborated human hallucination. Which is more likely?

Also it really does seem like you've moved the goalposts a lot here without really giving me a substantive response.

[27183]

To say that LLMs' existence is evidence for emergent phenomena in LLMs is tautological. I'm merely suggesting if you want to make a claim about emergence it would be best, especially in absence of a convincing theory, to demonstrate it experimentally. Otherwise probably better not to claim it's actually happening.

[sudb]

> To say that LLMs' existence is evidence for emergent phenomena in LLMs is tautological.

This is not at all what I was saying. I think you've already conceded that LLMs demonstrate emergent behaviour but you dismissed it as a "close enough facsimile to intelligence". I was saying that the emergent behaviour is reliably replicable, in response to your following statement:

> Figure out how to make it happen reliably and in such a way that you know it's actually happening as opposed to just something you're imagining.

I think there is real work underway in the area of interpretability. In the meantime, there appears to be plenty of empirical evidence for the claim that LLMs exhibit some sort "intelligence" in the enormous penetration that agentic coding has achieved in software development? Do you deny the usefulness of LLMs here, or are you going to assert that actually software development requires no intelligence of any sort?