[bem94]

I have only read a few passages (and some of the excellent quotes others have shared here), but I find the underlying message here so much more compelling than those found in the various "manifestos" which come out of Silicon Valley.

I think reading this helps me imagine a version of the future I'd actually like to live in. A version where technology is used well (rather than preaching for abstinence from technology) and where values other than "intelligence" (in whatever guise) are on an equal footing.

Even writing that makes me feel naive (and to an extent I know it is) but I think it would be inconsistent for someone who cheers for humanity's efforts to solve/chip away at "impossible" problems (like LLMs were thought to be not so long ago) to shirk from the challenge of making the world better for _everyone_.

[bluesign]

The thing is why that this feels so good future is; it is a system with no constraints. A bit like Star Trek universe in Roddenberry's imagination. This kind of utopia can only be achieved with all honest actors, but in reality systems are usually designed around bad actors.

Even with all morally good actors locally, there is no guarantees for external forces. Thinking it hypothetically, even with global coordination ( all good actors ) there is not a proven path that would lead us to better place from any starting point from past.

[energy123]

It's probably more predictive to model actors as being neither good nor bad but constrained by various collective dilemmas, such as prisoners dilemma, the security spiral, tragedy of the commons, race dynamics, collective action or first mover problems, information asymmetries, the commitment problem, among others. Those are the hardest problems to solve because they're pathologies that result from the global, largely amoral structure rather than consequences of the individual exercise of morality.

In the AI case, each firm is in an arms race, and nobody can slow down without effectively collapsing due to positive gross margins only being viable with a frontier model that attracts marginal demand. An appeal to morality might have an impact but more effective action would be to address the structure that the AI companies are situated in that causes this dynamic in the first place. In practice, thats going to be a global agreement to slow down, and global regulations.

[bluesign]

Yeah but this is a system problem; if we had this utopic system from the beginning we would not even have AI probably.

[datsci_est_2015]

This sort of rationalization of evil is a core of technocratic support for Trumpism, I find, and has parallels to the evangelical prosperity gospel. Choice tenets:

  - Fuck you, got mine
  - If I don’t do it, someone else will
  - Might makes right
  - Greed is good

It’s always cloaked in a veil of realism, but it’s just the classic 14-year-old-boy-just-got-introduced-to-the-prisoners-dilemma situation. There’s nothing philosophically interesting about it.

Ironically, these are often the same people denouncing multiculturalism, yet the culture they strive for is completely morally bankrupt.

[estearum]

And it's funny because the "realism" has been proven wrong over and over and over again for millennia. People do all sorts of selfless and generous things all the time! The entire premise is trivially disprovable by just going and asking a neighbor for some help with something.

That's not to say we should be naive about greed or malice existing or being powerful motivators (especially the former), but it is obviously not true that they're the only forces at play and therefore you are "just doing the logical thing" by succumbing to them. It's just the more destructive version of the same naiveté.

[datsci_est_2015]

Seems you and I have together struck a nerve. Maybe our sentiments would have been better received in an alternate subthread, but it’s all I could think about while reading the parent / cousin comments.

I haven’t read the full Magnifica Humanitas yet, but I would be pleasantly surprised if he touched on not just dehumanization of the other, but dehumanization of the self. Expanding on your thought, succumbing to those forces under the guise of just doing the logical thing is in a way self-dehumanization - to believe you are only capable of the “logical” thing instead of the moral thing.

[bluesign]

This was not my point at all. Maybe I could explained better, but main criticism I have is: you can bundle together objectives ( which are inherently good ) and create an utopia. But those cannot always be achievable.

Everything in life in trade-offs. Simple example is speed/quality/cost. I can tell easily:

- services should be cheap - services should be fast - services should be high quality

Now I created an utopia. Obviously this is amazing to listener. They agree. But is it achievable?

It is not saying greed is good or might makes right. But system means you need to construct from this ideals best outcome ( which comes at some trade offs)

[hammock]

Your idea and its (perhaps unsatisfying to you) resolution can be summed up easily by John Quincy Adams’ quote:

“Duty is ours. Results are God's”

[bluesign]

Yeah but my point is sometimes suboptimal parts can give better overall results. ( Considering also bad actors )

[datsci_est_2015]

Yeah there were probably more appropriate subthreads to have responded to. My point wasn’t quite neatly directed against yours.

[mapt]

To get from here to Roddenberry's communism, according to Roddenberry's lore, we passed through the Eugenics Wars, the Second Civil War, and then fifty years of World War Three and the 'post-atomic horror' before coming to our senses.

[snikeris]

> I think reading this helps me imagine a version of the future I'd actually like to live in. A version where technology is used well (rather than preaching for abstinence from technology)

I believe the Amish figured this out over a century ago.

[aleph_minus_one]

> I believe the Amish figured this out over a century ago.

The Amish rather came to a different conclusion (which I don't want to judge on, but on which I nevertheless have a different opinion than the Amish).

[46493168]

What is that conclusion which differs from the post you replied to? The Amish are mindful about their technology adoption.

[aleph_minus_one]

> The Amish are mindful about their technology adoption.

The central idea concerning the Amish's relationship to technology is that only technology is allowed if it does not destroy their community.

My personal values are much less based on upholding a community, but rather are much more rooted in individual freedom and independence. This means that I (likely) come to very different conclusions regarding this class of problems than the Amish do:

For example, I am less opposed to various kinds of technology that Amish would likely consider as as "community-destroying".

On the other hand, I guess I am much more opposed to technology that can be used to surveil the user and/or makes the user dependent on the whims of big tech companies than I guess the Amish are (i.e. the Amish would likely consider this as a much smaller problem concerning which technology to allow vs disallow; as I wrote: by my understanding their central concern is which consequences some technology has for keeping their community together).

To give evidence for the previous point: (by my impression - I am not US-American) you will rather not find many Amish people at political rallys against surveillance laws. The people who attend such rallys typically also have strong opinions on which technology to use or not to use (just talk to such people who are very strongly opinionated :-) ), but - as I pointed out - these technology choices come from very different basic premises than those of the Amish.

[snikeris]

Yeah they probably wouldn't show up to a political rally because of this:

> Separation from Evil

> The community of Christians shall have no association with those who remain in disobedience and a spirit of rebellion against God. There can be no fellowship with the wickedness of this earthly world; therefore there can be no participation in the organizations, works, church services, meetings or civil affairs of those who live in contradiction to the commands of God (this may include Catholics and Protestants as well as other religions and pagans). All evil must be put away, including using weapons of force such as the sword and armor.

https://en.wikipedia.org/wiki/Schleitheim_Confession

[exceptione]

  > much more compelling than those found in the various "manifestos" which come out of Silicon Valley.

Whenever I hear these "tech overlords", I am always baffled at the total lack of culture, the absence of taste, the empty visions and the implied complete subjugation of humans to ideals of "efficiency" or "quick and easy". Maybe they would have been more interesting people if they had been brought up in beautiful towns and cities, if they had lived in a rich cultural environment instead of being raised as consumer of cheap and flashy pop culture. Maybe we should tax bad architecture, it gives me headaches but others might incur heavier damage.

As an aside, at least Trump is drawn to the grandeur of high culture from historical times, but he also doesn't understand a jota about aesthetics, and so the White House gets turned into a tacky gypsy-style abomination with one dollar ornaments.

[bombcar]

We lost the “liberal education” (not the political one, but the “freeing” classical one) and it’s starting to show.

When you compare the robber barons to Google and Meta it’s kind of embarrassing- they build massive empires of iron horses screaming across the world and covered cities in magnificent buildings (stations, libraries, etc). G&M built an empire of advertising and … not much else?

[exceptione]

Indeed. The current crop doesn't have an idea for what they hoard their billions, it's just...emptiness. I propose we explain the tech's attachment to Accelerationism as a profound boredom and lack of purpose. "What does it mean to be human"--they don't value that question. Peter Thiel got interviewed a month or two ago, and he could not be brought to say that he sees value in preserving humanity. He would rather turn himself into a robotic contraption to extend his life.

When power fears death, some strange things happens.

EDIT: link to the interview with Thiel <https://xcancel.com/rcbregman/status/2036113528126394834#m>

[bombcar]

I’m reminded (and apropos as the Pope quoted him) of Tolkien’s description of the “eternal life” the Ring gives to mortals, and how it’s … not so desirable in the end.

[jjtwixman]

Indeed It's far more necessary that the utter dregs of humanity (e.g. Peter Thiel) eventually die of old age. Or put another way the damage of mortality killing good people is more than offset by the good of it killing the worst people with the most power. Because in the end it's probably not going to be your sweet mother who will get to live forever, it'll be people like Peter Thiel. No thanks, for the good of our species.

[bombcar]

“Those who used the Nine Rings became mighty in their day, kings, sorcerers, and warriors of old. They obtained glory and great wealth, yet it turned to their downfall. They had, as it seemed, unending life, yet life became unendurable to them. They could walk, if they would, unseen by all eyes in this world beneath the sun, and they could see things in worlds invisible to mortal men; but too often they beheld only the phantoms and delusions of Sauron. And one by one, sooner or later, according to their native strength and to the good or evil of their wills in the beginning, they fell under the thraldom of the ring that they bore and of the domination of the One which was Sauron's. And they became forever invisible save to him that wore the Ruling Ring, and they entered into the realm of shadows. The Nazgûl were they, the Ringwraiths, the Úlairi, the Enemy's most terrible servants; darkness went with them, and they cried with the voices of death.” - from the Silmarillion but it’s echoed in LotR also. And even Bilbo complains of being “butter spread over too much bread”.

[gmllama]

This is why I like the term "Dragon Sickness." There's seemingly only innate compulsion and no real human thought behind the hoarding. It becomes its own end. I cynically lament that it's human nature for billionaires to exist but if that is true, couldn't they at least be more entertaining about it? Bezos and Musk could be bleeding each other dry to get to the next star system by now.

[muttonette]

Google makes phones and phones are somewhat good. Better search had some value for humanity. Meta has no redeeming qualities or achievements, other than helping Trump get into office and defeat Iran.

[aerodexis]

People become tech-overloads because they are blind to these sorts of beauties - and that'd be fine if it wasn't for the fact that we have collectively allowed these people to come to power and have fallen for their empty promises of freedom and liberation.