[PeterisP]

For your thought experiment, I'd assert that the key missing part is experimentation in the real world, as that is what acquires new information, not the complexity of human brains working together.

If you took millions of genius-level immortal humans with all the same Roman data but had them sit in a blank, empty room with their hands tied and simply discuss philosophy for eternity, I'm certain that they would not be able ever spit out a correct predecessor of modern science like atomic theory. Perhaps they could spit out billions of theories including the atomic theory as well, but they would have no data to presume that the atomic theory is more relevant than any other. Extensive information processing can squeeze out every last ounce of knowledge from some data, but anything that isn't in that data can't be acquired by mere thinking about it. On the other hand, if you gave some "LLM++" the ability to toy around with reality and attempt all kinds of experiments to test various hypotheses, then I wouldn't assume that it would not be forever stuck in the knowledge of the era it was trained in.

[chrbr]

Yeah, I like that improvement/clarification. Good assertion. Now I wonder if it changes my stance: are the path modern LLMs are on ever going to replicate this environment for acquiring new information that humans currently operate in?

[CamperBob2]

As usual in these "Yeah, but can AI do this?" threads, the answer is yes, it is already happening: https://www.space.com/mars-oxygen-ai-robot-chemist-splitting...