[plutonorm]

Completely disagree with 2. Look up AIXI. It looks and feels to me like a completely plausible and mathematically grounded theory of intelligence. One principle aspect of the theory is that it allows you to see that intelligence is merely compression. This is the origin of the Hutter prize. Compression is almost the same thing as prediction, look into the math. Llms are prediction machines. Predicting the next word isn't some stupid goal it's exactly what you need to do to build an intelligent machine.

[atoav]

Whether you are fine with 2 depends on how much you are willing to shrug any possible outcome off and say: "I guess it's fine".

And, more essentially, how much the rest of the company just shrugs it off and says: "I guess it's fine", so when it blows up you can defend with the famous defence my Nazi grandfather also told me: "But everybody else did it too."

Remember the Xerox printer that was swapping out (and therefore falsifying) numbers in their compression algorithm whenever you made a copy? Imagine we could bring this to all of our business processes, but in new and unexpected ways.

[jszymborski]

I'm failing to see what AIXI, a theoretical RL agent, has to do with LLMs.

I never said that predicting the next word was a stupid goal, just that it isn't AGI.