[gretch]

Definitely sounds like a plausible and fun episode.

On the other hand, real history if filled with all sorts of things being treated as a god that were much worse than "unreliable computer". For example, a lot of times it's just a human with malice.

So how bad could it really get

[1vuio0pswjnm7]

"Definitely sounds like a plausible and fun episode."

There were several original Star Trek episodes that explored this scenario. Not plausible. Actual.

"So how bad could it really get"

Watch Rodenberry's orginal Star Trek to get some ideas.

[vlovich123]

It’s important not to confuse entertainment with a serious understanding of the consequences of systems. For example, Asimov’s three rules are great narrative tools because they’re easy for everyone to understand and provide great fodder for creatively figuring out how to violate those rules. They in no way inform you about the practical issues of building robots from an ethical perspective nor in understanding the real failure modes of robots. Same with philosophy and self driving cars - everyone brings up the trolley problem which turns out to be a non issue because robotic cars avoid the trolley problem way in advance and just try to lower the energy in the system as quickly as possible vs trying to solve the ethics.

[bigyabai]

Yes. This is a component of media literacy that has been melted away by the "magic technology" marketing of the 2000s. It's important for people to treat these stories with allegorical white-gloves rather than interpreting them literally.

Gene Roddenbury knew this, and it's kinda why the original Trek was so entertaining. The juxtaposition of super-technology and interpersonal conflict was a lot more novel in the 60s than it is in a post-internet world, and therefore used to be easier to understand as a literary device. To a modern audience, a Tricorder is indistinguishable from an iPhone; the fancy "hailing channel" is indistinct from Skype or Facetime.

[Certhas]

Everybody shits on the trolley problem, until it gets to the question of forcing people to get vaccinated...

[vlovich123]

Doesn’t apply. Disease is a societal group problem. Part of the social contract of living in that society is vaccination. You don’t have to get vaccinated but you then don’t get to enjoy the privileges of living with others in the community.

This isn’t anything like the trolley problem. And yes, taking actions has consequences intended or otherwise. That’s not the trolley problem either

[1vuio0pswjnm7]

'Star Trek' franchise an homage to humanist philosophy (2010)

https://www.telegram.com/story/news/local/north/2010/06/22/8...

"Ms. Sackett, with the aid of film clips, said that "The Return of the Archons," from the original series, was a good example of how Mr. Roddenberry employed elements of humanism in his works.

In that episode, a planet's population follows, in a zombie-like manner, a mysterious cult-like leader, who allows no divergent viewpoints.

The society absorbs individuals into its collective body and the world is free of hate, conflict and crime but creativity, freedom and individualism are stifled.

Ms. Sackett said that "Archons," like other Star Trek storylines, warns how people can be controlled by religion. In the end, the viewer discovers the cult leader is actually a computer."

"[N]o divergent viewpoints" sounds like Stackoverflow and forums run by software developers in general. The behaviour of "developers" can be extremelly cult-like.

Creativity, i.e., new work that is not comprised of a recombination of old work, does not seem compatible with "AI". The later relies on patterns found in old work.

[dmonitor]

"as bad as it can get" is somewhere in the realm of universal paperclips

[ben_w]

That will merely kill everyone.

"As bad as it can get" is an AI that, either by accident or due to malign influence, takes "I Have No Mouth, and I Must Scream" as a guide book.

Actually, I take that back, it would be what happens in the hells in Surface Detail.

[exe34]

Surface detail was one of those books that messed with my head a bit. A much worse example was Blindsight.

[TZubiri]

It exists https://m.youtube.com/watch?v=x0YGZPycMEU

[greesil]

Person god was not as scalable as AI god, so there's that.

[DrillShopper]

> So how bad could it really get

I don't know. How about we ask some of the peoples who have been destroyed on the word of a single infallible malicious leader.

Oh wait, we can't. They're dead.

Any other questions?

[gretch]

I’m saying this has happened multiple times in human history already.

How does doing it with a computer add anything?

[DrillShopper]

Look at the British Post Office scandal - "the computer is always right".

Say what you will about a human, but unless you're a religious zealot or blind you generally don't believe the leader to be infallible. But through the magic of silicon you can shut people up more effectively.

This makes computers an accelerator of the problem, and therefore warrants caution any time their output may be relied upon for life and death decisions.