[yosame]

Scott absolutely analyses topics in bad faith, particularly when it comes to politics or movements/people he personally supports. Here's one where he essentially says 'no study is ever fully representative, therefore my friends twitter polls are exactly as good evidence as a scientific study':

https://www.astralcodexten.com/p/selection-bias-is-a-fact-of...

[Sankozi]

Your comment is an example of analyzing in bad faith.

In the linked article there is a whole section that starts with "Selection bias is disastrous if". He definitely does not says anything remotely close to "twitter polls are exactly as good evidence as a scientific study".

[yosame]

It's essentially the thesis of the post. I will say the "exactly" part was a bit hyperbolic.

He starts by stating that people say his surveys and Aella's Twitter polls (which he calls surveys) have selection bias, meaning the conclusions don't count.

He then goes on to say (paraphrasing): "people say surveys have selection bias, but don't think real scientific studies have selection bias" and in the next paragraph says "but scientific studies do have selection bias and that's ok." Note that in this section, he is implictly relating scientific studies with his and aellas surveys.

In the section that starts with "selection bias is disasterous if", he gives a narrow view of why and when it's bad: "only if you're doing a poll or census that should include everyone". He neglects to say the ways in which it's bad for literally all science (it's a massive problem in drug RCTs, but he literally uses RCTs as a point for why selection bias isn't so bad later on).

In the next few paragraphs he says that selection bias is fine for correlations, and gives some examples of when scientific studies have selection bias that we accept (I'm not going into details here because the post is long, and I'm on my phone).

So he's said why selection bias is present in science, and that it's only really bad for census's and polls. He then concludes, "hey guys stop saying surveys are bad because selection bias, science is also biased".

In this post he starts by bringing up a critism of Aella's 'research', which she uses data from Twitter polls for. He then implicitly equates surveys and scientific studies, and says "look, science is biased too, so selection bias isn't bad". He then concludes, "so stop saying surveys are a bad because of selection bias".

He might not literally say "twitter polls are exactly as good evidence as a scientific study", but the point of the post was to detract from the valid criticisms against his and Aella's posts that use survey/twitter poll data by saying "science does it too".

[mercutio2]

His thesis is “you should not discount twitter polls with large sample sizes out of hand as totally meaningless, or if you’re going to, you need a better reason than this long list of issues that twitter polls share with RCTs.”

The contrapositive of his thesis is “there is some interesting signal worth paying attention to in large twitter polls, selection bias is in many unexpected places so be skeptical about sweeping conclusions even in RCTs”.

Your characterization of this as “so twitter polls are just as good as RCTs” seems wildly uncharitable to me.

[yosame]

I'll definitely admit to some bias here, I've been reading Scott's content for 7ish years and I've gone from loving his content to being increasingly skeptical about his writings.

Something I feel Scott (and rationalists in general) tend to do is obfuscate their points in large walls of text, that allude to their actual reasons for writing without directly stating it. Aside from the beginning and end of the post, he never mentions the context or specifics, and just talks about broad counterarguments against selection bias in general. He doesn't bring up relevant specifics like "Aella does polls about sex to an audience she cultivated by doing porn", which seems like an obvious source of bias to me. Instead he gets the best of both worlds, if someone criticizes internet surveys he can point them to the post, without including any specifics that can be easily argued against. It requires a similarly sized wall of text to respond, which most people won't bother with. I personally used to argue that's just how he writes and it's not intentional, but stuff like the email leaks made me change my mind on that [1].

I do wish I'd picked a better example though, what I'm really trying to say is that Scott is definitely capable of analyzing a topic in bad faith. He has his biases like the rest of us, and they get infused into his writing.

[1] https://imgur.com/a/gWeIK6c