[bluefirebrand]

> Move fast and break things!! Even when its the society you live in.

This is the part I don't get honestly

Are people just very shortsighted and don't see how these changes are potentially going to cause upheaval?

Do they think the upheaval is simply going to be worth it?

Do they think they will simply be wealthy enough that it won't affect them much, they will be insulated from it?

Do they just never think about consequences at all?

I am trying not to be extremely negative about all of this, but the speed of which things are moving makes me think we'll hit the cliff before even realizing it is in front of us

That's the part I find unnerving

[feoren]

> Do they think they will simply be wealthy enough that it won't affect them much, they will be insulated from it?

Yes, partly that. Mostly they only care about their rank. Many people would burn down the country if it meant they could be king of the ashes. Even purely self-interested people should welcome a better society for all, because a rising tide lifts all boats. But not only are they selfish, they're also very stupid, at least in this aspect. They can't see the world as anything but zero sum, and themselves as either winning or losing, so they must win at all costs. And those costs are huge.

[brobdingnagians]

Reminds me of the Paradise Lost quote, "Better to rule in Hell, than serve in Heaven", such an insightful book for understanding a certain type of person from Milton. Beautiful imagery throughout too, highly recommend.

[Henchman21]

> Do they just never think about consequences at all?

Yes, I think this is it. Frequently using social media and being “online” leads to less critical thought, less thinking overall, smaller window that you allow yourself to think in, thoughts that are merely sound bites not fully fleshed out thoughts, and so on. Ones thoughts can easily become a milieu of memes and falsehoods. A person whose mind is in the state will do whatever anyone suggests for that next dopamine hit!

I am guilty of it all myself which is how I can make this claim. I too fear for humanity’s future.

[Dracophoenix]

> Are people just very shortsighted and don't see how these changes are potentially going to cause upheaval?

> Do they think the upheaval is simply going to be worth it?

All technology causes upheaval. We've benefited from many of these upheavals to the point where it's impossible for most to imagine a world without the proliferation of movable type, the internal combustion engine, the computer, or the internet. All of your criticisms could have easily been made word for word by the Catholic Church during the medieval era. The "society" of today is no more of a sacred cow than its antecedent incarnations were half a millenium ago. As history has shown, it must either adapt, disperse, or die.

[bluefirebrand]

> The "society" of today is no more of a sacred cow than its antecedent incarnations were half a millenium ago. As history has shown, it must either adapt, disperse, or die

I am not concerned about some kind of "sacred cow" that I want to preserve

I am concerned about a future where those with power no longer need 90% of the population so they deploy autonomous weaponry that grinds most of the population into fertilizer

And I'm concerned there are a bunch of short sighted idiots gleefully building autonomous weaponry for them, thinking they will either be spared from mulching, or be the mulchers

Edit: The thing about appealing to history is that it also shows that when upper classes get too powerful they start to lose touch with everyone else, and this often leads to turmoil that affects the common folk most

As one of the common folk, I'm pretty against that

[andrewl]

Exactly. It was described in Chesterton’s Fence:

There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.”

[unsui]

I've called this out numerous times (and gotten downvoted regularly), with what I call the "Cult of Optimization"

aka optimization-for-its-own-sake, aka pathological optimization.

It's basically meatspace internalizing and adopting the paperclip problem as a "good thing" to pursue, screw externalities and consequences.

And, lo-and-behold, my read for why it gets downvoted here is that a lot of folks on HN ascribe to this mentality, as it is part of the HN ethos to optimize , often pathologically.

[jmount]

Love your point. "Lack of alignment" affects more than just AIs.

[chasd00]

Humans like to solve problems and be at the top of the heap. Such is life, survival of the fittest after all. AI is a problem to solve, whoever gets to AGI first will be at the top of the heap. It's a hard drive to turn off.

[bluefirebrand]

In theory this is actually pretty easy to "turn off"

You flatten the heap

You decrease or eliminate the reward for being at the top

You decrease or eliminate the penalty for being at the bottom

The main problem is that we haven't figured out a good way to do this without creating a whole bunch of other problems