[techostritch]

This method seems to lean into the idea of LLM as fancy search engine rather than true intelligence. Isn’t the eventual goal of LLMs or ai that it’s smarter than humans. So I guess my questions are:

Is it plausible that LLM’s get so smart that we can’t understand them. Do we spend like years trying to validate scientific theories confabulated by AI?

In the run up to super-intelligence, it seems like we’ll have to tweak the creativity knobs up, like the whole goal will be to find novel patterns humans don’t find, is there a way to tweak those knobs that get us super genius and not super conspiracy theorist? Is there even a difference? Part of this might depend on whether or not we think we can feed LLM’s “all” the information.

But in fact, assuming that Silicon Valley CEO’s are some of the smartest people in the world, I might argue that confabulation of a possible future is in fact their primary value. Not being allowed to confabulate is incredibly limiting.

[seeknotfind]

Yes, I agree, scientific theories have their value from being new/confabulated. Though this isn't mutually exclusive from solving today's problems of confabulations. The proposed methodology could be used to mark semantic coherence, but it doesn't mean we have to hide confabulations.

[pornel]

LLMs are language models, and I think it's best not to try to extrapolate them to general intelligence. They are universal language translators, and a lossy database of a lot of text. They might be a component of some bigger AI system in the future, but themselves they are not as intelligent as their marketing implies.