[Udo]

No. Contrarian anecdotes are good. They may turn out to be without merit, but then again so may the article itself. Having a real discussion is a good thing.

Also, I would like to propose a logarithmic scale for weighting such things. Say, if the article in question found something extraordinarily significant with 100 out of 100 samples resulting in A, then it's still not rational to weigh a contrarian viewpoint resulting in B with 1/101 - it should maybe be closer to 1/3 or something.

Consensus culture and worship authority are not desirable in my opinion. Arguments should be weighed on their merits and it's appropriate to explore other viewpoints or explanations even if they turn out to be dead ends most of the time.

[loup-vaillant]

Let's take an example, since they seem to work best: vaccines.

Vaccines protect you from the risk of contracting particular diseases, some of which are crippling, lethal, or incurable. Plus, most are extremely effective: once you take your shots, you are effectively immune. That's good.

There is a downside however. Sometimes, vaccines have side effects. Most side effects are quite benign, but if you're unlucky enough, they can be crippling, lethal, or incurable. That's bad.

From a medical point of view, vaccines are a net good (let's leave aside logistic considerations, or the effort required to go to the doctor). When you look at the stats, you stand a much better chance at life and health if you take the shot. Even for relatively minor illnesses like the flu.

Now, let's say someone post a heartbreaking comment about how her 9 year old daughter died of a vaccine shot, with all the gory details about the suffering, how she couldn't participate in her school's festival, the size of the coffin… I'm quite sure there are stories of the kind. Given the sheer amount of readers here, maybe one of you will more or less directly relate to that. My apologies to those who do.

Nevertheless, what makes a good story doesn't necessarily make good evidence. When you know of reliable statistics, and you read a contrarian anecdote, you should shift your belief in the direction of the anecdote by a precise amount, which is almost always tiny. What your brain will actually do behind you back however, is shifting your belief by a significant amount, often crossing the "reasonable doubt" line. That's not rational, but that's what will happen. Nameless statistics feel abstract, remote. An anecdote on the other hand feels concrete, real, close. Worse, you can spend far more time reading about the salient anecdotes than learning about the end results of reliable, but boring, scientific studies.

Another example: you don't win the lottery. Period. You don't know of any close family of friend that ever did. But maybe one of you readers do. Maybe that one could comment and say "Hey, but my cousin did win the lottery!". Would that prove me wrong? Not at all. It's just that when the sample size is huge enough, even the tiniest chance can actualize.

[retrogradeorbit]

Are you actually arguing against giving anecdotes weight by using... anecdotes?

[loup-vaillant]

Yes, I'm evil.

Here is the "good" version: Bayesianism is correct. Those who don't believe me may want to read E. T. Jaynes[1] or Eliezer Yudkowsky[2] (long, and may feel abstract and dull). But countless studies about biases showed that we humans are poor at correctly assessing evidence at our disposal. Some of those studies showed that some failure modes come from anecdotes. Downvoting seems harsh, but it's the best we currently have to combat those failure modes.

Now we don't want to overdo it. I suggest we put a comment citing which reliable statistics contradict the downvoted anecdote. Maybe that'll help avoid groupthink. We may also want to allow people to just say they have anecdotal evidence to the contrary of whatever.

[1]: http://bayes.wustl.edu/etj/prob/book.pdf

[2]: http://yudkowsky.net/rational/technical

[slurgfest]

Those are hypotheticals or thought experiments, not anecdotes... they would be anecdotes if they were claimed to have actually happened.

I'm not arguing against anecdotes, but there is an important distinction there

[archangel_one]

Those are examples, not anecdotes.

[loup-vaillant]

Yes, but I did make use of strong emotional story-like elements to make my point.

[JackFr]

Those aren't anecdotes, they are hypotheticals. They are useful, but statistically they occur even more infrequently than anecdotes.

[pbreit]

I didn't see any anecdotes in the post.

[the_bear]

So you're suggesting that because some people are bad at weighing information, anything based on personal experience should be downvoted? Successful entrepreneurs should no longer give advice? Programmers shouldn't weigh in on programming issues unless they're citing third-party research? Marketers can't suggest tips for how to promote a startup because it's possible their methods aren't universally successful?

[loup-vaillant]

It's just not some people. It's everyone. Including you, me, and those who know about the various failure modes.

And please don't straw man me. Personal experience is mostly great. Successful entrepreneurs may have better decision making processes, not just more luck. Programming issues should be weighted on, since there are so little reliable studies here, and the field is so young. Etc. I was just talking about the cases when the evidence that contradicts the anecdote is solid and definite.

[JoeAltmaier]

Disagree. Successful entrepreneurs are lottery winners with attitude, by and large. The programming field is no longer young, we should give up that old excuse. And no you are not being straw-manned, the argument is. Good to not take things personal here.

Studies are necessarily narrow and context-laden, even 'solid and definite' ones. The suggestion to automatically downvote anecdotes is too broad, and should be refuted.

[loup-vaillant]

> Successful entrepreneurs are lottery winners with attitude, by and large.

Possibly. Actually I don't know. Anyone knowledgeable should disregard my opinion.

> The programming field is no longer young, we should give up that old excuse.

Right. However, I don't feel like we're anywhere near clearing the chaos around the psychology of programming. I still don't know for instance why so many people cannot understand functional programming, which I personally find simpler than procedural programming in most cases I deal with. Or why technical debt doesn't seem to be taken seriously. Programming is several decades old, but it still feels young to me.

> And no you are not being straw-manned, the argument is. Good to not take things personal here.

Hmm, yes, I was too aggressive here. Sorry.

> Studies are necessarily narrow and context-laden, even 'solid and definite' ones.

Ah, I didn't think of this danger. You're right, we at the very least need safeguards. Like, tying downvotes to reasons why they happen, so we (high karma users, moderators?) may be able to nullify those which turn out to be bogus. But that's complicated.

Or, maybe we could just not downvote, but point out in a reply that this is contrarian anecdotal evidence?

[JoeAltmaier]

Yes; it would be wonderful if comments could be categorized. Something like meta tags, created by the community or automatically even. I would like that; it would be like an extra conscience telling me "This guy was making an observation, not an argument; target your response correctly".

[bad_user]

A vaccine is a virus in latent form, forcing your B-cells to produce antibodies for it. So yeah, personally I never take a vaccine that hasn't been in circulation for some time.

Also, considering that medicine is at the stage of alchemy and that doctors simply have no idea what long-term effects these vaccines have on our immune system, some questions do have to be asked.

Like, isn't it possible that with the prevalence of vaccines, our own capacity for generating antibodies gets affected?

And remember here that an exaggerated response of the immune system may be even worse than a lazier response. Such an exaggerated response may even kill you (e.g. Influenza). So either way, the long-term effects of over-reliance of vaccines may be quite bad.

[ars]

> A vaccine is a virus in latent form

Not exactly, it's not a virus in latent form, it's either a killed virus, a piece of a virus, or a different virus that is weak, but provokes the same reaction as the more important one.

(Do you know what latent means? It means that it shows up later, which vaccines do not do.)

> So yeah, personally I never take a vaccine that hasn't been in circulation for some time.

Yah, me too, but let's not overreact with nonsense.

> Like, isn't it possible that with the prevalence of vaccines, our own capacity for generating antibodies gets affected?

No, it's not possible. That's completely ridiculous. Do you know anything about vaccines at all? Seriously, that really makes no sense whatsoever. A vaccine does not do anything at all to our capacity to generate antibodies. All it does is take the exact same virus you would get if you got sick, and expose you to it in advance, that's all. It gives you a head start in making antibodies, but does not affect the generation of them in any way.

> And remember here that an exaggerated response of the immune system may be even worse than a lazier response. Such an exaggerated response may even kill you (e.g. Influenza).

And a vaccine creates a muted response, quite the opposite. Compared to a simple cold a vaccine consists of a minuscule number of virus particles. The entire trouble with making a vaccine is trying to get enough of a response, most of the time the body ignores it.

> So either way, the long-term effects of over-reliance of vaccines may be quite bad.

And how do you figure that? I'm not following your logic at all. Unless your logic is that the vaccine somehow changes the bodies response, which it doesn't. So hopefully now that I've cleared that up you will no longer claim this.

[bad_user]

On "latent" ... English is not my primary language and this was just a bad translation.

     Do you know anything about vaccines at all?

I guess not, but I can't help to not get worried about the rise of autoimmune disorders and I haven't heard yet a plausible explanation for the phenomenon.

     Unless your logic is that the vaccine somehow changes
     the bodies response, which it doesn't.

And how in the world would you know that?

[ars]

> but I can't help to not get worried about the rise of autoimmune disorders and I haven't heard yet a plausible explanation for the phenomenon.

The most plausible explanation is the http://en.wikipedia.org/wiki/Hygiene_hypothesis see also http://blogs.scientificamerican.com/disease-prone/2012/02/15...

> And how in the world would you know that?

How could it? If a vaccine could cause such a change so could any illness. A vaccine is just a piece of virus put where your body can notice it. Everything after that is entirely from the body.

For example rabies: Lethal right? But the body can actually clear the rabies virus with no trouble - almost. The trouble is that by the time the body gets rids of the virus it's too late.

So what do you do? You give the body the rabies virus ahead of time, and you do it in a way that prevents the person from actually getting sick. Then next time the body encounters rabies it's ready.

All vaccines work exactly this way: You let the person encounter the illness ahead of time. You make no change whatsoever in the person - all you are doing is making them slightly sick, but in a way that doesn't kill them.

Whatever change the vaccine causes, the illness also does - except the illness also causes damage as the virus replicates.

[phren0logy]

>Also, considering that medicine is at the stage of alchemy and that doctors simply have no idea what long-term effects these vaccines have on our immune system, some questions do have to be asked.

What the hell are you talking about? There is probably no single more life-saving intervention in medicine than vaccines. It is true that a small number of people have a bad reaction to them, but more people have a bad reaction to tetanus.

Those who do not vaccinate are risking re-emergence of preventable epidemics: http://www.sciencebasedmedicine.org/index.php/whooping-cough...

[tomjen3]

If some people have bad, even fatal, reactions to vaccines shouldn't the physicians first duty (do no harm) mean that they should separate the people who would suffer sideffects from those who wouldn't?

[Confusion]

No, not if 1 in 10000 people suffers that reaction, while there are equally negative consequences for 2 or more of those 1000 people if they don't get the vaccine and there is no known way to detect who will have the bad reaction.

And the ratio is much worse for actual vaccinations. You don't want to see what not vaccinating kids against polio results in...

[tomjen3]

I have seen the pictures, and the interviews.

But those are hardly data and the kids who got the live version (due to a fuck up) are hardly better of.

[phren0logy]

In the US, we do a screening for risk factors for reactions, and review information needed to give informed consent.

[jd]

I'd say that contrarian anecdotes are good if -- for instance -- the discussion spawned by the anecdote leads to a better understanding of the subject at hand. However, because anecdotes are stories we tend to remember them vividly and they affect our judgment disproportionally. Human brains are very flawed, and it may very well be the case that the negative effect of contrarian anecdotes vastly outweighs the benefit we get from the discussion it spawns. I don't know the answer. But I do know that we can't state with certainty that "having a real discussion is a good thing".

As for how we should weigh new evidence, this is essentially a solved problem: use Bayes' rule. Suppose that 100 out of 100 studies indicate that smoking is a leading cause of cancer and a contrarian viewpoint ("My grandfather smoked his whole life and lived to 123!") indicates otherwise. Then that anecdotal viewpoint should get approximately 0 weight. Zero. Nip. Zilch. Nada.

We're all experts in a few subjects at best. In those subjects we can easily explore different viewpoints, balance different arguments and keep track of the different schools of thought. We can even confidently diverge from expert consensus if needed. But in most subjects we're enthusiastic laymen at best. I don't think debates and exploration of different viewpoints then lead to much greater understanding. Just look at any forum on the internet (including this one). Debates aplenty and the few knowledgeable people get drowned out in a sea of contrarian musings.

Expert consensus is just the aggregate opinion of those who have the best understanding. So when a layman disagrees with the experts he's almost certainly wrong. What I see is the opposite of consensus culture. I see a willingness to disagree with the experts before understanding the subject material in depth.

[aptwebapps]

> ... a contrarian viewpoint resulting in B with 1/101

Well, see, that's part of the problem. The original sample of 100 was, hopefully, selected at random. Whereas the anecdote was selected by the person telling it because he or or she thought it was apropos. With a large enough population of potential commenters, the chances of someone doing so gets really high.