[Timwi]

> how do you take the shape of the human corpus and all its gradients and [somehow] arrive at something greater than human, where was the missing information hiding?

Well, how do humans do it? Scientists discover new stuff that isn't in any corpus. Even I as a lowly computer user occasionally figure something out about a software without reading a help screen. It's obviously possible to arrive at new information by interpolating existing information.

[morpheos137]

yes and it is imposible to verify and evaluate appropriately such information without empiricism. Any empricism LLMs show is stylistic mimicry not a hard coded operational constraint. You can prompt an LLM to test its claims but what it is really doing is still genrating plausible completions not following a proceedure. So of course new things can be discovered. The point is for them to be useful requires iterative real world grounded refinement and or subject matter expert judgment. The error is assuming scaling magically turns a prediction algorithmn into a cognitive agent that can exceed its masters. it doesn't. even if llms generate profound insights accidentally by definition if such insights are not in the corpus they are not retained given frozen weights and if beyond the human capability envelope the epistemically blind llm has no way to ensure retention if they arise during training.