[hiAndrewQuinn]

The idea that most AIs are unsafe to non-AI interests is foundational to the field and typically called instrumental convergence [1]. You can also look up the term "paperclip maximizer" to find some concrete examples of what people fear.

[1]: https://en.m.wikipedia.org/wiki/Instrumental_convergence

It's unfortunately hard to describe what a safe AI would look like, although many have tried. Similar to mathematics, knowing what the correct equation looks like is a huge advantage in building the proof needed to arrive at it, so this has never bothered me much.

You can see echoes of instrumental convergence in your everyday life if you look hard enough. Most of us have wildly varying goals, but for most of those goals, money is a useful way to achieve them -- at least up to a point. That's convergence. An AI would probably get a lot farther by making a lot of money too, no matter what the goal is.

Where this metaphor breaks down is we human beings often arrive at a natural satiety point with chasing our goals: We can't just surf all day, we eventually want to sleep or eat or go paddle boarding instead. A surfing AI would have no such limiters, and might do such catastrophic things as use its vast wealth to redirect the world's energy supplies to create the biggest Kahuna waves possible to max out its arbitrarily assigned SurfScore.

[robertlagrant]

I couldn't find concrete examples that weren't actually of AI with godlike powers.

[ben_w]

What do you mean by "godlike powers"?

We flatten mountains to get at the rocks under them. We fly far above the clouds to reach our holiday destinations.

We have in our pockets devices made from metal purified out of sand, lightly poisoned, covered in arcane glyphs that so small they can never be seen by our eyes and so numerous that you would die of old age before being able to count them all, which are used to signal across the world in the blink of an eye (never mind (Shakespeare's) Puck's boast of putting a girdle around the earth in 40 minutes, the one we actually build and placed across the oceans sends information around it in 400 milliseconds), used to search through libraries grander than any from the time when Zeus was worshiped, and used to invent new images and words from prompts alone.

We power our sufficiently advanced technology with condensed sunlight and wind, and with the primordial energies bound into rocks and tides; and we have put new πλανῆται (planētai, "wandering" star) in the heavens to do the job of the god Mercurius better than he ever could in any myth or legend. And those homes themselves are made from νέος λίθος ("neolithic", new rock).

We've seen the moon from the far side, both in person and by גּוֹלֶם (golem, for what else are our mechanised servants?); and likewise to the bottom of the ocean, deep enough that スサノオ (Susanoo, god of sea and storms) could not cast harm our way; we have passed the need for prayer to Τηθύς (Tethys) for fresh water as we can purify the oceans; and Ἄρης (Ares) would tremble before us as we have made individual weapons powered by the same process that gives the sun its light and warmth that can devastate areas larger than some of the entire kingdoms of old.

By the same means do our homes, our pockets, have within them small works of artifice that act as húsvættir (house spirits) that bring us light and music whenever we simply ask for them, and stop when we ask them to stop.

We've cured (some forms of) blindness, deafness, lameness; we have cured leprosy and the plague; we have utterly eliminated smallpox, the disease for which शीतला (Seetla, Hindu goddess for curing various things) is most directly linked; we can take someone's heart out and put a new one in without them dying — if Sekhmet (Egyptian goddess of medicine) or Ninkarrak (Mesopotamian, ditto) could do that, I've not heard the tales; we have scanners which look inside the body without the need to cut, and some which can even give a rough idea of what images the subjects are imagining.

"We are close to gods, and on the far side", as Banks put it.

[southernplaces7]

Wonderfully written, and though I've seen this kind of reshaping of perspective on our human achievements in the modern world before, you've done it exceptionally well here.