[carnagii]

Human cognition is riddled with exploitable defects. Biologically we are basically just highly pretentious and neurotic monkeys. All of human history is full of people looking for someone to blame for their condition (gods, devils, spirits, corporations, etc) but it never changes.

Keep in mind, from an evolutionary perspective we are exactly the same people who were burning witches at the stake and throwing people in lakes to determine their criminal culpability a few hundred years ago. We just have a different set of superstitions and delusions now.

[xyzzy123]

I get it, is that a thing you want to fix though, or to prevent? Education or regulation or something else, if we take as a given that the market has failed?

I wrote a comment twice (and deleted it as inflammatory, it’s a fact about reality that is largely counterproductive to debate) where I pointed out that Donald Trump is the president of the U.S. and it’s wonderful or terrible depending on your point of view. I don’t want to derail all conversation into irrelevance but what if the “wrong thing” is what people want? What do we even want anyway? Is “exploitation of cognitive errors” just a thing we say when people decide they want things we think are dumb?

The true horror: what if this is the least worst thing people want?

[whatshisface]

How can you tell the difference between your own personal hangups and absolute moral truths? That's what we're really debating here. Good luck solving that one in an afternoon...

Option one, check with your community. That's the approach the pro-censorship side is taking, they only propose censoring content that every acceptable silicon valley individual thinks is objectively harmful. They also support drowning witches. Wait, no, wrong culture, they got it wrong, we got it right this time we promise.

Option two, check with the conscience of the accused. Human beings regret most of their decisions, and people are their dumbest in the heat of the moment, and further people tend to dig in when pushed from the outside, so maybe the path to virtue can only be people improving themselves. However if you're convinced that your enemies are all a bunch of complete evil psychopaths then that won't work because clearly psychopaths don't do that.

Option three, optimize for something completely unrelated, and pretend to be motivated by whatever suits your goals - if morals are "in," then pretend to be moral. That's probably what's going to happen if we can't pick between 1 and 2.

[AnimalMuppet]

Option four, have an external standard that is neither based on the community nor the individual. But the problem is, that standard has to be right. If it's wrong, then it's going to be used to censor things that contradict it, and therefore enforce the wrongness. So you need a clear standard of what's right. Christianity once furnished such a standard for the West, but no longer. The closest we have now is the law, but that's halfway to option one.

[whatshisface]

>So you need a clear standard of what's right.

How do you verify it? It's no easier than before.

[carnagii]

> is that a thing you want to fix though, or to prevent?

wanting to fix is pretty obvious. capable of fixing is a different story. biologically there is no fix (genetic engineering maybe? but that is super sci-fi). so we can try to correct with technology or social conventions but those fixes never change the underlying biological defects so how effective can they really be?

Look at the social engineering aspects of organized Christianity. How well did those work? Look at the social engineering aspects of the U.S. experiment like universal education and literacy. How well did those work?

It is a little bleak but in reality universal literacy has been a complete failure (in the U.S. at least). Probably 80% of the population is at the level of what used to be called "knowing your letters" but they are functionally illiterate (they have never done any significant amount of reading in their life, and aren't really capable of it). That is not a popular opinion, at least not for public consumption, so we can't even begin to address the issue because we refuse to acknowledge that it exists.

[pesmhey]

If you include the notation used in math and science, then the actual level of literacy is at the same levels as you would have seen 500 years ago. It’s actually kind of similar; the liturgical class was literate enough to read and study the Bible, and the masses relied on them to parse the information. Today, it’s statistical models, but the concept is the same.

[nine_k]

As a side note, what always amuses me when I see phrases like "what people want" is that it sounds like there are all the people, and the speaker is not them.

I suppose the situation is usually more complex; in the simplest case it's just "us vs them", but likely it's a more detailed separation of groups, based on culture, values, etc. Thinking a bit more explicitly about that, and especially about who are "we", the ingroup, could be beneficial.

[mcphage]

> I get it, is that a thing you want to fix though, or to prevent?

Do you think technology should be used to help us become better? Or help us become worse?

[kazagistar]

What is this that you refer to exactly? The deep non uniformity of our current society and politics, globally, indicates that there is, at the very least least, a lot of room to maneuver and improve. Liberalism, for example, likes to push the narrative of the least bad option, but often it does so to reject ideas that have a clear precedent of working better than the ongoing decline of equality and flight of alienated workers towards reactionary politics we have now.

[aswanson]

That's a rather misanthropic definition of human beings.

[mcguire]

History is rather misanthropic.