[nomel]

> without a large chunk of that being pre-existing information.

Is there any evidence that novel reasoning is present in LLM? I've never been able to make that work, and I believe Apple's paper some time ago was good evidence that it doesn't exist. In my experience, sparse latent spaces result in a complete, comical, failure in reasoning.

[drivebyhooting]

See the new mathematical proof published by OpenAI.

I’m not very valiant to verify its veracity. But even if the math is merely derivative it merits mention.

[nomel]

True, but that's an unknown internal model, without details of the architecture. We'll have to see if the LLM model, itself, was responsible for the "novel" bits, or if it was stuffs bolted onto the LLM that made it possible. I suppose "LLM" is maybe no longer sufficient to describe the systems that LLM are being integrated into, so maybe my point is pedantic/semantic.