[bjt]

"frontier level" is doing a lot of work there, but the idea would be to only feed it earlier sources.

There are people working on this.

e.g. https://github.com/haykgrigo3/TimeCapsuleLLM

[lovecg]

The problem is the amount of data with that cutoff is really minuscule to produce anything powerful. You might be able to generate a lot of 1700s sounding data, you’d have to be careful not to introduce newer concepts or ways of thinking in that synthetic data though. A lot of modern texts talk about rates of change and the like in ways that are probably influenced by preexisting knowledge of calculus.

[NewJazz]

Doesn't it prove GP's point then, that LLMs themselves simply aren't capable of creating/proving new theories and axioms?

[codebje]

Without passing opinion on GP's point, I think that just proves it's hard to establish a data set that doesn't bias toward the result you're hoping to find.