[skurilyak]

"Contrary to the more fantastic predictions for AI in the popular press, the Study Panel found no cause for concern that AI is an imminent threat to humankind" -- Stanford Study Panel, comprised of seventeen experts in AI from academia, corporate laboratories and industry, and AI-savvy scholars in law, political science, policy, and economics

[threepipeproblm]

What immediately follows is rather sobering, too: "No machines with self-sustaining long-term goals and intent have been developed, nor are they likely to be developed in the near future."

[zardo]

What about corporations? They are self-sustaining non-human information processing systems with long term goals, and they are subject to selection pressure. They are rudimentary humans-in-the-loop artificial life.

[TheOtherHobbes]

I think the most likely route to destructive AI is a corporation with an AI CEO - possibly one that takes over from a human CEO in a boardroom coup.

Corporations already have legal personhood and act in their own interests. It's going to be much easier to automate and formalise business decision making than to develop a true general intelligence with a full spectrum of human characteristics.

This may sound like science fiction, but as competence increases shareholders - who typically are only passingly interested in moral issues - are likely to demand the increased returns an AI CEO can bring.

[dragonwriter]

> Corporations already have legal personhood and act in their own interests.

But AIs don't, and corporate officers must be persons (natural persons, even -- a corporation can't be a corporate officer.)

[cLeEOGPw]

A person could be made a mediator of AI to overcome it.

[zardo]

Many CEOs are already just trying to operate as a share price optimization algorithm. They don't choose to inject human values into their organizations.

[yourapostasy]

> ...operate as a share price optimization algorithm...

That in itself is not so bad, as extremely long-timeframe constraints (say, >50 years) upon such an algorithm could conceivably be consonant with current decision-making behavior that externalizes many input costs (employee overtime, environmental damage, etc.). Running the algorithm to pay out in very short timeframes (a month to a year) due to most CEOs' anticipated short tenure is what seems to cause undesirable optimizations.

[oldmanjay]

That you don't share their values does not make them non-human

[js8]

That they are human doesn't mean they cannot be a threat to humankind. ;-)

[js8]

I was thinking about a scenario where some crazy billionaire would build an autonomous AI that would operate a fleet of secretive hedge funds and financial trading companies. The AI wouldn't have any need to know what these companies - staffed by real people - are actually doing. It would just try to maximize profit and it would just kill the companies that would deviate too far from the norm, and build another companies in their place. No CEO of any of the companies would ever see this "investor" in person, everything would be sent over email, and done to the billionaire's name. The billionaire would eventually die and those companies would continue to operate completely autonomously.

[ThomPete]

This is the major problem with the line of reasoning.

A tiger doesn't need to be self aware or have intent to be dangerous.

[soperj]

Pretty sure the tiger has intent on eating something...

[ThomPete]

Sure but it doesn't need to be as smart as humans to hurt us and neither do AI it just needs to be everywhere.

https://wiki.lesswrong.com/wiki/Paperclip_maximizer

[the8472]

Are you thinking about grey goo?

The paperclip maximizer generally refers to a AGI with a value system that is not aligned with humans. An AGI smart enough to achieve its goals (making paperclips) so efficiently that it becomes a threat to humans through sheer resource consumption.

So it's not a good example of dumb-tiger-AIs occasionally becoming a threat to humans, which still on average are able to outcompete a tiger with ease.

[ThomPete]

I am thinking about systems set up to protect us can end up hurting us because in order for them to be helpful they get so much power over us that their continued improvements become fundamental to our survival beyond the scope of our ability to understand it.

The "system" does not need to have intent or be even closely aware to be dangerous to humans.

And so the article sets up a false premise if the quoted conclusion is to be the base of judging whether it's going to be a threat to us or not.

[ThomPete]

Understanding robotics does not mean you understand exponential growth, it's very very hard to understand even for those who believe they do.

http://assets.motherjones.com/media/2013/05/LakeMichigan-Fin...

5 years ago it was thought to take decades to beat a human in Go.

Just 10 years ago self driving cars was something you joked about.

We consistently overestimate progress in the short run and underestimate in the long.

[argonaut]

"I believe that a world-champion-level Go machine can be built within 10 years" - Deep Blue architect, 2007.

http://spectrum.ieee.org/computing/software/cracking-go

[Inlinked]

And for the self-driving cars:

http://people.idsia.ch/~juergen/robotcars.html

[aab0]

You forgot the rest of that quote: "...based on the same method of intensive analysis—brute force, basically—that Deep Blue employed for chess."

[argonaut]

He was half-wrong about the specific methods used (I say half-wrong because half of AlphaGo relies on relatively-brute-force MCTS). I don't think this detracts from my point - it is hard for any researcher to predict the exact methods that will be used a decade from now.

[ThomPete]

You don't have a point.

You tried to use one expert as a refutation of my claim that there was a general consensus with how long it would take for computers to beat a human in Go.

A simple search on google will provide you with plenty of writing backing that sentiment up.

Asking any of your friends who took AI classes back then would confirm the same.

[argonaut]

Please keep your rant to the other subthread. I was addressing aab0's point.

[ThomPete]

You are missing the point completely.

Of course there were people who believed that Go would be able to beat humans. Just as today there are people who believe that AI can be a treat.

But it wasn't a majority of people who believed Go would be able just as it isn't a majority who believes AI can be a threat.

I.e. just because the majority believe something doesn't mean it will be so (or vice versa)

[argonaut]

You're going to need citations for your unsubstantiated claims. I've brought forth a highly prominent expert opinion in 2007 that computer Go would be dominant by 2017.

And not media reports from present-day that just repeat this meme that almost everyone believed Go wouldn't happen for decades.

[ThomPete]

Again you are missing the point.

A highly prominent expert opinion but the general consensus was that it would take a long time. Ask anyone who went to AI class back then.

Here is another expert

"In May of 2014, Wired published a feature titled, “The Mystery of Go, the Ancient Game That Computers Still Can’t Win,” where computer scientist Rémi Coulom estimated we were a decade away from having a computer beat a professional Go player. (To his credit, he also said he didn’t like making predictions.)"

http://www.wired.com/2014/05/the-world-of-computer-go/

You also find highly prominent expert opinions that AI is going to be dangerous and experts who don't believe it. Most people don't believe it, most people believe robots wont take jobs either.

And no I don't need to provide you with anything since you have only problem my point that most people didn't believe it would happen which is why you didn't link to anything saying that most believed it would happen.

[Jach]

The real irony is that Rémi said that in 2014. Sometime around then if not before deep learning was showing it could knock down more and more problems, it was pretty clear to anyone who was keeping up that if someone figured out how to combine deep learning with Rémi's work on monte-carlo tree search they ought to end up with a powerful go bot, perhaps even pro-beating. What took me personally by surprise was that the development (which also required a pretty large army of GPUs, though I wondered if we might see specialized hardware like deep blue) was done mostly out of the public eye, not even tests against humans on go servers, until suddenly it was announced the bot beat a 3p.

I think it may be rare that you see consensus on those sorts of "it's imminent, someone just has to do the work" problems because it requires simultaneous knowledge of multiple developments, and knowledge doesn't always disseminate as fast as it takes one group to just do the work. Now I'm remembering this related maxim: http://lesswrong.com/lw/kj/no_one_knows_what_science_doesnt_...

[reacweb]

I think Rémi's quote shall be take in its positive meaning: given the already impressive rate of progress in computer go before 2014, go programs would succeed to reach professional go player in around 10 years. When suddenly arrives a gorilla like google with its money, resource and expertise in AI, these 10 years have reduced to 2.

[argonaut]

There is irony in trying to refute my single expert opinion with another single expert opinion (via an article).

Reply to below: You're the one asserting that experts thought Go wouldn't be dominated by computers for a long time. The burden of proof lies on you. "Some experts thought it would happen by now, some didn't, there was no consensus" doesn't have quite the ring to it!

[Animats]

"Just 10 years ago self driving cars was something you joked about."

That changed on the second day of the 2005 DARPA Grand Challenge. Suddenly, there were lots of self-driving cars running around. The sudden change in the attitude of the reporters there was remarkable.

[maxerickson]

Self driving cars were serious research more than 10 years ago.

https://en.wikipedia.org/wiki/DARPA_Grand_Challenge

Several vehicles finished the 2005 course. The one that finished first won a $2 million prize.

[ThomPete]

Yes but it wasn't even remotely seen as possible by the masses for decades to come. It's already here now.

[guelo]

Cool! I wasn't aware. Where I can go buy one?

[unprepare]

http://www.curbed.com/2016/8/31/12691516/self-driving-bus-ve...

you can at least ride in one (if youve got bus fare)

[chclau]

Are you aware that in Singapur, driver less cabs are already in service? And that many of the biggies have plans for autonomous cars within a few years?

[ThomPete]

http://www.forbes.com/sites/andrewbender/2016/08/31/self-dri...

[daveguy]

That's a link to an about a company who is testing autonomous taxis. You could also link to Google's self driving car tests. Neither are ready for sale to the general public.

[guelo]

"It’s not that the cars are fully driverless, either"

[riboflava]

AGI progress is unlikely to be limited by some exponential curve that we're just not far enough along though. Rather it seems more limited by some key insight no one has found yet. Sure we'll be able to retrofit some exponential curve onto it after the fact, since when it appears it will change everything drastically. But this is in contrast to e.g. the human genome project which started out with the expectation of a specific exponential curve to give an estimate for project completion. Your more general point I agree with, however, it's not enough to dismiss the report.

[cLeEOGPw]

People also predicted there would be fully functional human replacing kitchen assistants by the year 1960. Just because someone overestimates or underestimates things, it doesn't affect how slow/fast it may progress. So adjusting future predictions based on offset of past predictions just makes no sense.

[at-fates-hands]

>> We consistently overestimate progress in the short run and underestimate in the long.

You can't overestimate the economic pressures on progress as well.

Remember in 2007 when everybody thumbed their noses at hybrid and electric vehicles in the US? Ford was still pumping out record numbers of their behemoth Excursion model.

Then the economy crashed, and people suddenly needed a fuel efficient car and then they all traded in their SUV's for what? Toyota Prius' which were an after thought a few years prior - in the span of 18 months, Toyota couldn't keep them on the lot.

I can see one or more catastrophic disasters where there is a sudden need for AI to rescue the human race in some capacity. Think nuclear war, environmental disaster, biological catastrophe, etc.

[gnahckire]

Have you heard of AI winter?

https://en.wikipedia.org/wiki/AI_winter

Too much opportunistic thinkinking caused crazy hype cycles.

[daveguy]

Can you name anything in nature that experiences exponential growth, or is it all logistic growth?

[ThomPete]

Technology is part of nature in my view. I don't distinguish. If you mean in biology then I guess cell division is an example?

[daveguy]

No, I meant in nature. In all of the natural world. Physics through economics and everything in between including technology. (Hint: nothing is exponential -- it always levels off. Otherwise we would have been consumed by it.) I would be genuinely surprised and extremely curious to see any natural phenomenon that maintains exponential growth.

I guess compound interest could be considered indefinitely exponential, but you eventually reach a barrier in what's insured and it is a relatively small exponent. Also, is it still savings if you never spend it? I wonder what is the longest continuous account in banking that has never been touched.

Anyway, that is a tangent and a somewhat artificial scenario. Can you name a naturally occurring scenario? I would accept technology if you could show that it isn't going to level out like all other natural phenomenon.

[njbooher]

The size of the universe.

[cLeEOGPw]

Bacterial growth is exponential until a limit is reached. You can find many examples. Doesn't really have anything to do with the article or OP post though.

[ThomPete]

I am not sure I understand your point.

Who says it needs to be exponential forever?

[PhasmaFelis]

Pretty much every living thing reproduces exponentially until it reaches the carrying capacity of its environment.

(Humans, fortunately, are starting to get better about this.)

Also, you want "logarithmic," not "logistic."

[wging]

> Also, you want "logarithmic," not "logistic."

Don't think so - https://en.wikipedia.org/wiki/Logistic_function

The 'S' curve of logistic growth looks exponential for a while, which is why the question arises. By contrast, no one mistakes logarithmic growth for exponential growth for very long.

[Strilanc]

https://en.wikipedia.org/wiki/Logistic_function#In_ecology:_...

[Houshalter]

Key word "imminent". They are talking about the near term future. AI risk people are talking about risks in 30+ years or so.

That said, it's typical for even experts to underestimate progress in rapidly advancing fields. No one predicted AI would be so good by now, say 5-10 years ago. Now computers are beating Go and rivaling human vision.

[vhold]

I think it's implied throughout all their predictions only apply up to 2030, so I think that is a fairly safe bet.

Giving this the headline of "One Hundred Year Study.." was confusing. That's the name of the ongoing effort to do this kind of analysis, but the paper is named "Artificial Intelligence and Life in 2030"

[skurilyak]

My original headline was: "Stanford Releases 28,000-Word Report on Artificial Intelligence (AI)", but one of the admin changed it to: "One Hundred Year Study on Artificial Intelligence: 2016 Report"

[wildmusings]

The machines have clearly made a deal with the reptilians and infiltrated Stanford. :)