[hamner]

That is important, but extremely challenging to execute for two reasons.

1. Historically, technology has outpaced the legal framework. Look at copyright law, software patents, and internet commerce for some examples.

2. It is next to impossible to predict the impact that "Strong AI" or a "Singularity" would have on society. Science fiction literature is filled with thousands of different scenarios. Do we expect Congress or another legislative body to create a framework based on each potential manifestation of strong AI, on the 0.01% chance of it occurring in the next 10 years?

Food for thought - there's a chance that the first implementations of Strong AI occur as a result of a public and government-sponsored research program. There's also a chance that they will come about by a small team of dedicated researchers who will use the technology to their (or its) own ends, legalities and ethics be damned.

[dodo53]

Yes, I think my current thinking is: this should be considered. But I'm also cynical that lawmakers could get it right, or that the laws would be followed. Strong AI would presumably have military applications too, or could allow rogue nations to outcompete economically if they had 'friendly' strong AI.

As, as long as most of the population / electable people are religious, probably the idea cannot be debated (as many religious would say consciousness cannot be simulated).

[patrickk]

"Strong AI would presumably have military applications too, or could allow rogue nations to outcompete economically if they had 'friendly' strong AI."

If you wish to explore this further, you may like to read the books behind the Halo video games, they're surprisingly good. For example, one of the books has an AI character, a cowboy, who controls the agricultural systems on a food-producing planet called Harvest.

Of course, there's also Cortana, one of the main characters from the Halo universe:

http://en.wikipedia.org/wiki/Cortana