[richardhod]

Do please note that Stross is quite well known not to a fan of the idea of the Singularity, and that his book should not be taken as a manual for the future, and rather as a cautionary tale and fantastically creative techno future

[radarsat1]

> his book should not be taken as a manual for the future, and rather as a cautionary tale and fantastically creative techno future

Frankly a lot of sci-fi that is intended as the latter is often taken as the former. I feel like there is some kind of lesson there: if you create it, expect someone to take it the wrong way.

If you are depicting a dystopia, be aware that there is a selection of people out there that will consider it a utopia from the point of view of the worst characters in the book and say to themselves, "that sounds awesome, I'm going to build that."

[loxias]

I think you are (depressingly?) correct. c.f. Xi's China. I had no idea... In stories after stories I've heard from friends who escaped I kept thinking "this is just a cheap rip off of 1984".

Of course, none of them had read 1984...

Edit: With apologies for a lack of citation, there is a growing problem in the functional design and layout of government and law enforcement command offices. Upper management had brought in a design firm to make a new counterterrorism center, and they looked to Hollywood as inspiration. As a result, day-to-day operations were impaired, because of course that's not what real work looks like. There's no "huge screen" covering a whole wall or everyone using touchscreens.

[harperlee]

Mind you that showcasing that to potential clients, as sad as it is, might well make up in marketing for the lost efficiencies.

Edit: just saw this is not a private company, but you get the idea: the room is effective in selling an image of spectacular efficiency and oversight.

[joe-collins]

> Some of you might assume that, as the author of books like "Singularity Sky" and "Accelerando", I attribute this to an impending technological singularity, to our development of self-improving artificial intelligence and mind uploading and the whole wish-list of transhumanist aspirations promoted by the likes of Ray Kurzweil. Unfortunately this isn't the case. I think transhumanism is a warmed-over Christian heresy. While its adherents tend to be vehement atheists, they can't quite escape from the history that gave rise to our current western civilization. Many of you are familiar with design patterns, an approach to software engineering that focusses on abstraction and simplification in order to promote reusable code. When you look at the AI singularity as a narrative, and identify the numerous places in the story where the phrase "... and then a miracle happens" occurs, it becomes apparent pretty quickly that they've reinvented Christianity.

> Indeed, the wellsprings of today's transhumanists draw on a long, rich history of Russian Cosmist philosophy exemplified by the Russian Orthodox theologian Nikolai Fyodorvitch Federov, by way of his disciple Konstantin Tsiolkovsky, whose derivation of the rocket equation makes him essentially the father of modern spaceflight. And once you start probing the nether regions of transhumanist thought and run into concepts like Roko's Basilisk—by the way, any of you who didn't know about the Basilisk before are now doomed to an eternity in AI hell—you realize they've mangled it to match some of the nastiest ideas in Presybterian Protestantism.

> If it walks like a duck and quacks like a duck, it's probably a duck. And if it looks like a religion it's probably a religion. I don't see much evidence for human-like, self-directed artificial intelligences coming along any time now, and a fair bit of evidence that nobody except some freaks in university cognitive science departments even want it. What we're getting, instead, is self-optimizing tools that defy human comprehension but are not, in fact, any more like our kind of intelligence than a Boeing 737 is like a seagull. So I'm going to wash my hands of the singularity as an explanatory model without further ado—I'm one of those vehement atheists too—and try and come up with a better model for what's happening to us.

https://www.antipope.org/charlie/blog-static/2018/01/dude-yo...

[Mistletoe]

Second reference to Roko’s Basilisk I’ve discovered today. Fair to say the AI wants me to learn this today. Let’s see if my life becomes superintelligence AI torture from this point on. I’ll do everything I can to prevent it from being created, if so. :D

[Firmwarrior]

The trick to avoiding AI turning your life into a living hell is to just stay off Twitter, TikTok, and most of Reddit

[Animats]

> I think transhumanism is a warmed-over Christian heresy.

There's some of that. "Uploading" is going to be really tough. Probably harder than strong AI. If you can build weakly godlike AIs, why bother uploading obsolete biobrain content into advanced hardware?

Although life extension is likely, in the form of editing DNA into a new longer-lived race. Retrofitting existing people is less likely to work.

I don't see strong AI in sight yet, but we're getting progress in the more "human" areas, like art. We may get Microsoft Middle Manager 2.0 before we get a robot that can do auto repair. Then we get "machines should think, people should work", and highly-profitable machine-run companies. This is perhaps followed by the Butlerian jihad.

There's been an implicit assumption in SF that AI capabilities would progress roughly in the direction of skills which lead to increased income levels in humans. That may have been a bad guess. Watch the public rhetoric change when the first machine CEOs outperform humans.

[dragonwriter]

> I think transhumanism is a warmed-over Christian heresy.

Christianity covers enough time and space that basically every idea that isn’t Christian orthodoxy is found in Christian heresy, so that’s probably true but not a ground for dismissal except from tbe perspective of Christian orthodoxy.