[fredfoobar42]

There's this attitude that because computer work on pure logic, anything a computer does must therefore be logical and rational. The problem is that the humans who program the computers will often encode their own illogical, irrational, biased, and flawed logic. The computer is not logical and rational, it just follows instructions well. If the instructions are illogical, irrational, biased, and flawed---but executable---the computer will follow them. And when they spit out a illogical, irrational, biased, and flawed result... well, it must be logical, rational, unbiased, and unflawed, because the computer is purely logical, rational, unbiased, and unflawed.

We code our biases into the algorithms we use every day. They inherit our flaws. The sooner we all wake up to this, the better.

[yummyfajitas]

On the other hand, if the instructions are clearly defined and machine "intelligence" is used, the computer will often fix the biases of it's creators.

https://www.chrisstucchio.com/blog/2016/alien_intelligences_...

What you are pushing is nothing but the idea of (secular) original sin for algorithms.

Unfortunately, innumerate reporters imagining AI as somewhere between a human and a 5 line python script doing exactly what the creator proposed are unable to comprehend this.

[gshubert17]

> We code our biases into the algorithms we use every day. They inherit our flaws. The sooner we all wake up to this, the better.

And our values. And assumptions. In Arthur C. Clarke's "2001: A Space Odyssey" the homicidal behavior of HAL was ultimately based on the conflict between his secret instructions and what he was able to share with the human crew, Bowman and Poole.

[dcole2929]

This is actually a serious concerns when it comes to AI. People tend to think it's overblown but non deterministic behavior is bad because you don't know what the program will do. Maybe it crashes, maybe nothing happens, maybe it nukes your cpu. You simply don't know. The first rule of true AI has to be to ensure it values human life. It would be grossly irresponsible to create an entity more intelligent than us and do otherwise. But what happens when the military inevitably decides these new AI things would make really good drone pilots. Or when a sufficiently powerful enough AI comes to the conclusion that the best way to protect human life is to take human life. What's the end result of that conflict. What's stopping a sufficiently intelligent AI from rewriting it's own code to get around restrictions it doesn't like. We already have self modifying code. It's a scary thought, and the fact that people are basing these things off the way humans think makes it even scarier