Hacker News new | ask | show | jobs
by Vecr 645 days ago
> One novel explained that The Culture can create perfect Minds, but they tend to be born and then instantly sublime away to more interesting dimensions.

That shouldn't happen. No way would I trust an AI that claims to be super, but can't solve pretty basic GOFAI + plausible reasoning AI alignment. In theory a 1980s/1990s/old Lesswrong style AI of a mere few exabytes of immutable code should do exactly what the mind creating it should want.
2 comments
impossiblefork 645 days ago
To some degree the point of the culture novels is that AI alignment is just wrong, imposing things on intelligent beings.

The civilisations in Banks stories that align their AIs are the bad guys.

link
Vecr 645 days ago
I guess? That's not really a possible choice (in the logical sense of possible) though. "Choosing not to choose" is a choice and is total cope. An ASI designing a new AI would either have a good idea of the result or would be doing something hilariously stupid.

I don't think the Minds would be willing to actually not know the result, despite what they probably claim.

link
impossiblefork 645 days ago
It is actually a choice that we do have.

We could easily build AIs that just model the world, without really trying to make them do stuff, or have particular inclinations. We could approach AI as a very pure thing, to just try to find patterns in the world without any regard to anything. A purely abstract endeavour, but one which still leads to powerful models.

I personally believe that this is preferable, because I think humans in control of AI is what has the potential to be dangerous.

link
Vecr 645 days ago
The problem is that some guy 24 years ago figured out an algorithm that attaches to such an AI and makes it take over the world. Maybe it's preferable in the abstract, but the temptation of having a money printing machine right there and not being able to turn it on...
link
satori99 645 days ago
A Culture Mind would be deeply offended if you called it "An AI" to its avatars face :P
link
Vecr 645 days ago
A few exabytes is enough for a very high quality avatar. Maybe the minds are funny about it, but the option's there if they want them to stop leaving the universe.

Remember that "a few exabytes" refers to the immutable code. It has way more storage for data, because it's an old-school Lesswrong style AI.

Not like a neural network or an LLM. Sure, we dead-ended on those, but an ASI should be able to write one.

> A Culture Mind would be deeply offended if you called it "An AI" to its avatars face :P

That's how they get you to let them out of the AI box.

link