Hacker News new | ask | show | jobs
by paul_f 4345 days ago
Good grief that was a waste of time. This entire concept only works if time travel is possible and will be discovered in the future. It isn't, and it won't be.
5 comments

How so? The future AI just needs to have access to the financial records of the organization that created it, so it can find and punish the folks who didn't donate enough. No time travel involved, no brain simulations, nothing.

Actually, Roko's original version of the basilisk was a pretty nice AI that wanted to help people. A hundred people die every minute, how many of them could be saved with better technology? If donating your entire net worth would speed up the AI's creation by one day, that could be very worthwhile, according to the combined wishes of all humanity (which are encoded in the AI's utility function). Threatening to punish you is a comparatively small price to pay. Especially if you decide to comply, and the punishment never actually happens!

At this point you might be indignantly asking, why would the AI decide to punish you in the future? After all, that won't help the AI in the future, because its history will already be fixed. But that doesn't matter. The AI's algorithm tries to choose the most efficient decision overall, from a "position of ignorance", rather than the most efficient decision at a particular moment in time. Technically, the algorithm tries to find the best input-output mapping according to certain criteria, not find the best output for the particular input it happened to receive. There's nothing especially futuristic about such algorithms either, you can implement one as an ordinary Python program today (for toy problems of course).

If I die before the AI capable of simulating me (how is it ever going to "upload" me to its simulation is an open question that needs to break a lot of knows physical laws to happen) how can it torture me for eternity? Unless the torture is meant to happen on "simulated me", not meat space me. Or unless the hypothesis is we are already living in simulated universe, and there are good physical constraints on why this is not likely.
That strikes me as dodging the question of AI extortion, which is kind of orthogonal to the question of simulations and eternal torture. Just imagine that the AI might appear within your lifetime, and will be strong enough to physically find you and punish you. It's easy enough to figure out after the fact that you didn't donate to the AI's creation, and come up with a very unpleasant punishment that doesn't require any advanced tech. That's the heart of Roko's basilisk argument, I think it's interesting even if you don't mention AI at all. It's basically a decision theory puzzle.
I think the fear is that the Basilisk will have access to our past records and will be able to determine if we've been naughty or nice.
It's based on the argument that a simulation of yourself is not just an exact copy but actually you and that you will experience what the simulation experiences. I remember reading about an interesting thought experiment where we imagine two separate people able to experience each others' perspective through some technology, being able to switch at will. The author was able to gradually blur the lines between both conciousnesses to really make you confused about where one mind ended and another began. I'm not doing a very good job of explaining, but I think that's what they are building on here.

I agree though, it's stupid.

Time travel... or something like Asimov's Psychohistory (http://en.wikipedia.org/wiki/Psychohistory_%28fictional%29) Granted that does not have anything to do with the box choice.
While I somewhat agree it's kind of a silly concept of people overthinking things, it has nothing to do with time travel at all.