[raz32dust]

In the same vein, "Human intelligence" led to the 2008 crash. Human's aren't infallible either. My problem with this argument is that there is no logical basis for how exactly human intelligence is different. Computers aren't there yet in most domains, but there is no evidence to say they can't get there. How soon is anybody's guess right now.

While AGI might be far off, I can certainly imagine computers running larger and larger sub-worlds. e.g, if all cars were self-driving, I am reasonably sure we can design traffic to be more efficient with all cars coordinating with each other instead of humans trying to do so.

[jerf]

"My problem with this argument is that there is no logical basis for how exactly human intelligence is different."

I can give you a partial answer, which is that human intelligence is somewhat slower, which mitigates the ability to crash the entire economy in 15 seconds. That's why a stock market can crash in 15 seconds nowadays, but "the housing market" can't. This gives people some time to do some things about it with some degree of thought, rather than all the agents in the system suddenly being forced to act in whatever manner they can afford to act with 15 milliseconds of computation to decide.

[robbiep]

That’s also why there are trading halts built into all major exchanges, so if something triggers a large fall, there is a circuit breaker in the decision process of 18-48 hours for people to process and digest information

[gricardo99]

Just FYI, NYSE Rule 80B[1] halts trading for 15 minutes, or until the next day if the first 2 levels are breached. CME follows similar rules.

1 - http://wallstreet.cch.com/nyse/rules/nyse-rules/chp_1_3/chp_...

[greenyouse]

Just for fun here's a link to a 2010 flash crash before those rules were in place where a trillion dollars were dropped in half an hour due to algorithms.

https://en.wikipedia.org/wiki/2010_Flash_Crash

[scotty79]

Maybe if economy can be crashed in 15 seconds it should be crashed in 15 seconds do we can have a saner economy?

Letting AI in is like fuzzing for computer programs. It can break your program in interesting ways but that's a good thing.

[bartread]

> Maybe if economy can be crashed in 15 seconds it should be crashed in 15 seconds do we can have a saner economy?

How do you know that would be the outcome?

[adrianN]

Numbers getting smaller on the stock market doesn't destroy any real value. Companies still have all the IP they had before, all the real estate, all machines... If the bits in some computer's RAM are so disconnected from reality that they can change so dramatically without any catastrophe in the real world, then the rational choice is to disregard those bits.

[phildawes]

Crashed markets reflect a real difference in what people are prepared to pay for assets. That has a real direct impact on how people plan and behave.

[raz32dust]

Slower only because humans need more time to arrive at the same conclusions, and use more data points. Either one can be made true of computers too.

[tlb]

Qualitatively, humans are likely to say, "Whoa! Hold on. This doesn't make sense." when presented with unreasonable data, while algorithms are likely to proceed and do whatever.

We don't have good programming abstractions for "This doesn't make sense." You want something between proceeding and aborting.

[raz32dust]

Why not? Probability + threshold/activation function in neural nets is basically modeling that.

[new4thaccount]

I read once that a soldier in Russia was supposed to fire off nuclear warheads once when faulty sensors erroneously indicated that America had already done so.

Had the system been a mathematical contract, we would all be dead. Instead, a hero determined sat pondering whether he should do it and then realized the sensors had to have been defective as he wasn't dead yet.

[Zanni]

You're probably thinking of Stanislav Petrov: https://en.wikipedia.org/wiki/Stanislav_Petrov

[new4thaccount]

Yes and thanks for linking. I re-read it and according to the article he didn't actually have the power to press the button, but not immediately reporting to his superiors who had the "button" had the same impact and my previous statement remains true about automated systems built by imperfect beings.

[yters]

Humans don't think kittens are avocados with complete certainty because someone flicked a speck of green paint on the kitten.

[Viliam1234]

Before stating this with complete certainty, I would ask Oliver Sacks first.

https://en.wikipedia.org/wiki/The_Man_Who_Mistook_His_Wife_f...

[yters]

His patients realized something was wrong with their perception, unlike a computer.

[inimino]

Some of them did!

[raz32dust]

https://en.wikipedia.org/wiki/Proof_by_example

[exelius]

The biggest problem with “human intelligence” is that our memory is so short and influenced by emotion. Though judging by the amount of data created and lost over the last decade, that’s probably true of computers too.

[raz32dust]

But is emotion bad? Emotions can be thought of as programmed biases, and we might have evolved them because they are collectively good for the species. Emotion is what caused many of the examples stated in this thread where a human made a better decision than a computer.

[exelius]

I don't think emotion on its own is bad or good; but it's not something that a computer can emulate. You may get better decisions in some scenarios with certain stakeholders, and worse decisions in other scenarios.

When we talk about "programmed bias", we need to be careful because different subsets of the population hold different bias -- so the bias programmed in is reflective of the experiences of the team that developed the software rather than the population of users it affects.

[onemoresoop]

In the same vein, a machine should always be geared towards being a tool put into the use of the human intelligence. This tool was designed by human intelligence after all. The problem is when human intelligence creates such a tool that becomes so complex it no longer understands how it works, what it does and how to stop it.

There's no doubt that more things will be managed by machines and artificial intelligence but let's not make human intelligence a second class citizen.

[maxxxxx]

I think what will happen is that a few people at the top will manage the machines and benefit from them while they will be trying to make the other humans second class. I think a scenario like “Elysium” is very possible. That is until there is a revolution and then nobody knows how that will end.

[dwaltrip]

Without some robust general intelligence in the loop, whether that be AGI or human, computers will always have countless strange, unpredictable, and devastating failure modes, especially if given increasing responsibility.

This is the reason why few systems of significant complexity and risk exist today without humans in the loop.

[inflatableDodo]

>computers will always have countless strange, unpredictable, and devastating failure modes, especially if given increasing responsibility.

Sounds like most people I know. We are much better at reflecting on them though.