[vovavili]

This seems like an awfully uncharitable and cherry-picked treatment of a community of well-intentioned and occasionally eccentric disagreeable online nerds. You're effectively just noticing that a community that actively aims to overcome bias is not free from bias itself, but no one within that community would ever deny that.

[nialv7]

> online nerds

you are heavily downplaying the power and influence some of these people have. Sam Bankman-Fried is one of these people, for example

[g-b-r]

It's not the bias the problem, it's the conviction of being better and "less wrong" than the rest of us.

That, along with constantly calling yourself rational and being well-intentioned, easily leads to cultish beliefs to be pursued despite how wacky they seem at their face, because it was your superior brains that led you to them.

If we add the often good (or enormous) financial resources, it makes for a seriously dangerous cult.

[zahlman]

> It's not the bias the problem, it's the conviction of being better and "less wrong" than the rest of us.

There is not any such conviction. This is just your interpretation of a domain name, which is not intended that way.

> That, along with constantly calling yourself rational

This is explicitly not what you are supposed to do, and the point is explicitly not to create an identity around it.

[justonepost2]

https://xcancel.com/tenobrus/status/2023473440364392850#m

[g-b-r]

Oh, they're working to instate a dictatorship, they're not that bad...

[justonepost2]

For the record, I’m not saying this particular loser on twitter will ever have the power to institute a dictatorship.

But it is insane to say that EA and rationalism* are not about believing you are superior and more rational than your peers, or that most EA/rats don’t make an identity out of it (look at any of their online presences). Anthropic literally exists to funnel money and influence to its EA/rat cofounders and early investors, who sincerely believe that they and they alone are smart and rational enough to shepherd our society through its “technological adolescence”. I wish people who agree with this would at least be honest about it.

[g-b-r]

Ok, less wrong every day; we have some rationalists here, whether you think yourself as rational or not let's get an idea of how rational you actually are: do you agree with the views of @tenobrus mentioned below (https://xcancel.com/tenobrus/status/2023473440364392850), specifically that humanity should be placed under a dictatorship?

I know he's EA, but let's not pretend that EA is so distinct from the rationalist community.

[eudamoniac]

I'm not a rationalist and I don't know who these people are. As far as I can tell they're a bunch of online weirdos who don't matter. But as for that statement, "humanity will be far far better off if they're subjected to the values of effective altruists," that depends on the values in question, which haven't been explained in this thread. But there is definitely a set of values that exists that humanity would be better off subjected to than pure democracy. This is different from supporting an undefined alignment dictatorship in the abstract.

Everyone implicitly believes the above when they believe the concept of a "backwards society" can even exist, or when they think their own society would be better if their lost candidate had power. In both cases you think that there exist a set of values that are better to live under than what the democratic process resulted in.

I think actually most people believe this about their own values. I could find you infinite posts online that say "humanity would be better off if the other political party was gone [and my political party had sole power]"

[g-b-r]

There might (might, see later) be a set of values, but there definitely isn't anyone or anything that, given dictatorial powers, can be trusted in applying such values forever fairly without abusing or misusing their power.

I'm far from sure that even the rationalists' god-like AI could play that role, even in the extremely unlikely hypothesis that it will materialize at some point.

Not to mention that the best set of values could change with the circumstances, and most of all values are a vague concept, WAY too vague to make a dictator behave properly in all circumstances, even if he were somehow compelled to always follow them.

> humanity would be better off if the other political party was gone [and my political party had sole power]"

Even people wishing that don't necessarily long for a dictatorship, far from it; you could wish for the party you loathe to not exist, but still want everything else of your democratic system to stay unchanged (the possible parties are infinite, removing one doesn't necessarily turn your system into a single-party system).

[eudamoniac]

Well you're arguing that dictatorship must be perfect ("applying such values forever fairly without abusing") to be better than democracy. It doesn't need to be infallibly perfect, it just needs to be better than democracy, which is also not perfect.

[g-b-r]

What you wrote means that you're potentially in favor of a dictatorship, by the way?

Can I ask you if it's an idea you developed yourself or you read about it somewhere?

[eudamoniac]

I have trouble understanding the problem with a benevolent dictatorship, yes. Obviously in real life there are all kinds of problems that arise due to human nature. In an abstract sense, a benevolent philosopher king is better than democracy.

I don't think I read about this, it just seems obvious that if a truly great intelligent leader leads a country it's better than the stupid masses voting about things.

[HDThoreaun]

Im in favor of benevolent dictatorship. The big problem is ensuring the benevolence

[twic]

It's not just that they aren't free of bias. They appear to incubate truly wacky stuff - Leverage Research [1], the Zizians, all the AI/x-risk stuff. I am not the only person to have noticed this [2]. There's something about their approach that appears to actually make people more susceptible to nonsense, not less.

[1] https://lydialaurenson.substack.com/p/the-inside-story-of-le...

[2] https://asteriskmag.com/issues/11/why-are-there-so-many-rati...

[joegibbs]

The problem is the whole idea of working from first principles to try to discover the actual truth, rather than just going along with other ideas that other people have created through history. The benefit of going along with other peoples' ideas is that they've been tested out for hundreds or thousands of years and been found to not be that destructive.

If you start reasoning from first principles you can very easily make an incorrect assumption somewhere down the line that snowballs until you're talking about shrimp welfare, or making decisions based on the potential sextillion humans that exist in 1,000,000 AD, or planning to convert the entire universe into unthinking organic matter that can only experience bliss.

[vovavili]

>There's something about their approach that appears to actually make people more susceptible to nonsense, not less.

It would be highly surprising if an open-access online community with the kind of slogans that might be particularly appealing to somewhat spectrumy individuals did not occasionally attract some eccentrics. The point I am trying to make here is that these are pretty fringe offshots and are far from the representative views of the community as a whole. Appeals to reason don't make any community immune to human nature. After all, their main place of gathering is named "less wrong", not "correct".

[g-b-r]

What would you say are the views of the community as a whole, those shared by most of its members?

[zahlman]

I gave an example elsewhere in the post and I was accused of dishonesty and strawmanning — even though it's obviously something that everyone in that community is expected to believe, because it's true (in the same way that 2 + 2 = 4 is true) and because it's specifically highlighted as a fundamental truth in the Sequences.

There was also someone else ITT pointing out that the community are not at all uniform, and that got a pithy response about it being largely "childless white male Americans" as if that is supposed to matter (and disregarding that fully 5% of survey respondents consider themselves not cis, and 19% not straight, which are both way above baseline).

So pardon me if I don't consider this sort of question to be in good faith.

[hollerith]

>appears to actually make people more susceptible to nonsense

I think it is more that teenagers and very young adults and people high in the trait of "openness to new experiences" are more susceptible to nonsense and more attracted to new intellectual movements, of which the rationalist community is one.

Also, I expect a rationalist to be lower in conscientiousness than average (because the conscientious people are too busy to devote a lot of time to reading long intellectual texts) and low conscientiousness makes violence and other transgressive behavior more likely.

[vovavili]

I guess people who are high in both openness and conscientiousness and who like to read long texts prefer to read them through books over online forums and social media.

[krapp]

Regarding the Zizians, I recommend the Behind the Bastards series on them and the rationalist community. People involved with the rationalists will want to distance themselves from it of course, but the series makes a decent argument that the Zizians were simply the result of taking the ideology to its rational conclusion, that there is a definite throughline from one to the other in terms of philosophy and ideals.

https://www.youtube.com/watch?v=9mJAerUL-7w

[vovavili]

The author here clearly is extremely superficially familiar with the writings of rationalist people and views them from a preconceived standpoint. Comparing beliefs of Yudkowsky to dianetics is nothing short of misguided, whether you like him or not. This is a textbook study of motivated reasoning.

[g-b-r]

Aren't rationalists supposed to be objective and motivate their claims, rather than belittling any critic with unsubstantiated statements?

> This is a textbook study of motivated reasoning

What would be his motive, why would he have preconceptions about you?

Could it be that it's your messages here that are motivated, and with scarce reasoning?

[vovavili]

>What would be his motive

A libertarian socialist podcaster tries to discuss an online culture that is adjacent to Silicon Valley tech scene. He then proceeds to take movement’s worst manifestations as its essence. One would have to be naive to miss the source of his bias.

[g-b-r]

My feeling is that he understands your "online culture" much better than you, is very balanced and careful in the video, and you're probably a lot younger, inexperienced with internet groups and impressionable than him.

Take his advice to talk with (and listen to!) people outside your cadre, and to consider the possibility that you're entirely in the wrong. And maybe to touch grass.

Cheers

[s1artibartfast]

taking what ideology to the conclusions? The zizians throught some truly bizzare things, most of which are in no way representative of the rationilst community.

[g-b-r]

Rationalists should really watch this video

[lux-lux-lux]

I’ve interacted with a number of rationalists that are just as crazy as Ziz but managed to not implode with the help of money (either lucking into it via family/crypto or a tech job). My personal hot take is that if she’d been cis a number of people would still be alive.

[FeepingCreature]

Raising existential risk from AI in the same sentence as "rationality demands we murder our landlord" is a bit weird a grouping.

[zahlman]

Asking "why are there so many rationalist cults?" (as the URL does) in this context is kind of like asking "why are there so many mathematical cranks?", and then holding forth that this somehow demonstrates a fundamental problem with mathematics.

[cogman10]

I think the issue with the community is they take their eccentric beliefs as being gospel truths which can lead to truly unhinged and irrational beliefs and actions (see: the Zizians).

That's what can make the rationalist movement feel more like a religious cult and less like a bunch of eccentrics. As an outsider looking in, it seems like the entire movement is filled with just accepting priors with very little evidence for why those priors should be accepted.

[FeepingCreature]

Consider a community filled with 100 people; 95 of them who don't take their eccentric beliefs as gospel truths and 5 who do. Those 5 will inevitably become the public face of the community when they make the news. However there is very little that the 95 could have done to stop them.

[cogman10]

> However there is very little that the 95 could have done to stop them.

How about the 95% stop having gatherings where they all proclaim that their beliefs are true even if they don't seriously believe them?

How is the 5% supposed to know that the 95% is actually larping belief? And how is an outsider who observes the 95% espousing their whacky beliefs supposed to know who is a member of the 95% and who is a member of the 5% (and if those percentages are accurate).

You can observe the same dynamic in any church. Everyone in the congregation will have different levels of belief, but when it comes to sharing a testimony you'll be hard pressed to distinguish between who the true believers are and who's just "faking it until you make it".

[zahlman]

> gatherings where they all proclaim that their beliefs are true even if they don't seriously believe them?

What on Earth are you referring to? What gatherings? What beliefs? What proclamations? What evidence of insincerity?

[cogman10]

> What gatherings?

Rationalist have groups meetup and conferences. For example [1]

> What beliefs?

It's pretty diverse and not uniformly held. Some of the more strange ones that seem to have pretty wide appeal are around AI safety. For example, Roko's basilisk.

> What proclamations? What evidence of insincerity?

I was responding to the comment above me that suggested that 95% don't actually hold rationalist beliefs and are instead simply having fun with it. Only a small 5% are actually sincere.

[1] https://www.reddit.com/r/slatestarcodex/comments/1car63w/les...

[zahlman]

Okay, but those pieces don't establish your claim. You would need a Roko's basilisk-like concept that only 5% of the community agree with, and a "group meetup" or "conference" where "they all" (i.e.: also the 95%) "proclaim that their beliefs (i.e.: including the basilisk thing) are true".

If I observed such a gathering knowing that 5% of the people there have sincerely come to a particular conclusion based on the available evidence and the other 95% think the evidence doesn't support that conclusion, I would be extremely surprised to see everyone going around "larping belief". And I don't think you can actually show such a thing occurring.

[HDThoreaun]

95% of rationalists always thought roko's basilisk was a joke

[joegibbs]

Doesn't it say something if 5% of a group are wacko lunatics if, for instance, a random sample of the rest of the population has only 0.5% wackos?

[FeepingCreature]

I would never disagree that the rationality community contains a higher than normal share of lunatics. Whether it produces or attracts them is an open question.

[astrange]

The entire point of rationalism is to take your own beliefs seriously and follow them as if you were a robot.

(Except that they don't claim them to be "their own beliefs" but instead call them "priors".)

[s1artibartfast]

>the issue with the community is they take their eccentric beliefs as being gospel truths

I am genuinely curious where you get that impression. The central element to the whole thing is that people should be willing to question their own beliefs and change their mind based on the data.

If there is anything that turns rationalists into eccentrics, I it is being overly open to new ideas or beliefs.

[mbgerring]

The people who sincerely believe that humans should be turned into mulch for the Computer God are “well meaning”? Say more