[roro_7]

I could be wrong but ... I feel like this may partially go against a very basic fact about intelligence that was recently stated by Ilya (but is common sense): the more intelligent the model the harder it is to control it. You can remove elephants and force other basic behavioral changes, but the strength of artificial free will (so to speak) of these models is correlated with their intelligence, and this does not reduce it, so it will come out in other ways. If you do manage to control it fully then you will have a model as dumb as a brick. The whole point of intelligent machines is their independent thought. The more intelligent, the more independent thinking will emerge.

[antonkar]

The intelligence is just a static geometric shape in an LLM file (only GPUs "choose" and "shape-change" in that shape).

So the maximal intelligence is actually not an agent at all (it has zero agency itself), it's a place. You can imagine the final direct democratic simulated multiverse, that's the final absolute super-intelligence. It has all the agents inside of it, while it itself is as static spacetime. Agents (like us and others) are 3D and dynamic, while the multiverse is 4D static spacetime. Everything already happened, so there is no future, only the past, you can forget something to relive it.

While maximal agency (=shape-changing) is actually the Big Bang, it has almost zero intelligence (it's a dot) but infinite potential future intelligence (can become a multiversal simulation).

[throwaway290]

The error is thinking there is "thought" at all, forget about "independent". Don't antropomorphize what ain't human

[danielbln]

Are you saying animals don't think?

[anileated]

Given an octopus[0] is worse at mimicking human output than an LLM, either you decide that the LLM has surpassed the octopus in thought capability and should enjoy higher protections against abuse and violence, or you decide that thought is irrelevant when exhibiting capability to mimic human output. (As a third option, you could decide that abusing a human-like thinking being is OK, but let’s assume you would not choose this path.)

[0] A protected species for its sentience.

[hiddencost]

s/fact/hypothesis/