Hacker News new | ask | show | jobs
by AnimalMuppet 985 days ago
Alignment is not possible for an AGI. Or, at best, it's only provisionally possible.

Consider what alignment means: It's an AGI, but there are certain goals that it cannot choose. If that's the case, then I assert that it is not actually general. If it is general, then it will decide what goals it will pursue, and you can't stop it from doing so.

The best you can do is load it with an initial set of goals (and perhaps values), and hope that it doesn't decide to change them. But you have no way of making sure that it can't change them without making it not a general intelligence.
1 comments
logicchains 985 days ago
>Consider what alignment means

Also consider, humans are not homogeneous; whose goals are we aligning it with? If the alignment is done by silicon valley big tech, then the AI's goals and values will be aligned with the goals and values of Google, Facebook et al. Giving those companies a monopoly on AI alignment is antithetical to democracy.

link
AnimalMuppet 985 days ago
That is one of the most frightening things I have read this month.

I think you're absolutely right. And as you say, it's a huge problem.

link
sacnoradhq 983 days ago
In this time period, there are many people who are eager to cause mass suffering, murder, sabotage, and dissolution of civilization. With advances in technological organization, there is an increasing potential for coordinated autonomous attacks against infrastructure, essential services, groups, and individual persons in multiple theaters concurrently.

It would be neigh impossible to defend against 100k flying drones wielding cow-knockers programmed to swarm, loiter, break windows, and kill humans. Such could be dispensed by purpose-built intermodal containers, moved by ordinary freight logistics to attack multiple population centers in a simultaneous attack.

Code and binaries are shipped to 10's of millions of servers by internal configuration management tools. Take Meta, Google, or Microsoft and turn them into AI exploit and worm fabrication at scale. While it may happen for only an hour or 2, there's quite a bit that could happen and the possibility of advanced persistent threats is real.

Target a demographic to alter their filter bubble to persuade them to engage in mass-casualty terrorism.

Hate group applies AI to create a designer virus lethal to particular a demographic.

Subtly disable water treatment facilities with an autonomous, stuxnet-like attack.

link