Hacker News new | ask | show | jobs
by andrewflnr 894 days ago
Not a great analogy. Today we have all kinds of profit-driven companies "going to the moon" without thinking too hard about the risks. There is not, and practically can't be, a central safety effort that has more effect than releasing reports. No one is enforcing quarantine.

If there was life on the moon in an analogous scenario, it would be a matter of a few trips before it was loose on earth.
2 comments
mastersummoner 894 days ago
Yes, but that's today. When the moon landing initially happened, nobody had ever been to another celestial body before, whereas now we have lots more experience visiting them and sampling their atmospheres and surfaces.

Nobody's ever created AI before, so we're in a similar situation in that nobody has firsthand experience of what to expect.

link
wavemode 894 days ago
The specific part of the analogy that breaks down is that nobody actually knows if, when, nor how we will ever create AGI. So all safety efforts are necessarily speculative (because the field itself is speculative).

Like, if the scientists working on the moon landing didn't even know yet what the moon was made of, nor whether they would be getting to the moon by slingshot, by elevator, by rocket, by wormhole, or by some other yet unknown means, it would be very hard to make any meaningful proposal for how we would stay safe once we did get there.

link
andrewflnr 894 days ago
Oh definitely, that part of the analogy works fine.
link
cubefox 894 days ago
Sounds like a great analogy?
link
andrewflnr 894 days ago
Does it? Do you think "pressing on" is the optimal course in both scenarios for the same reason?
link
cubefox 894 days ago
In both cases a central safety effort seems nearly impossible. E.g. trying to enforce international AI risk cooperation via air strikes against data centers [1] can easily be avoided by defecting countries by building supercomputers underground.

With the moon bugs this wasn't a big problem, as they were so unlikely. But for AI the risk seems quite large to me.

[1] https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no...

link
andrewflnr 894 days ago
Right, but the original comment is trying to draw comfort from the actual Apollo scenario where central enforcement very much happened, not from my modified scenario. I think we're on roughly the same page.
link