[lutusp]

A quote: "There seems to be a simple way to instantly increase a person’s level of general knowledge. Psychologists ..."

Ahh. Psychologists. Expectations calibrated.

"Expecting to know the answers made people more likely to get the answers right."

Ah, yes -- that conclusion should be easy to rigorously quantify, explain in neuroscientific terms, turn into a general theory, and replicate before anyone assumes we're doing actual science. But no one will shape a theory, there will be no replications, and this study, like 99% of psychology studies, will disappear without a trace, only to be inadvertently repeated years from now by someone who will arrive at the opposite conclusion.

[pradocchia]

explain in neuroscientific terms

Isn't that putting the cart before the horse? The article draws attention to a phenomenon that isn't predicted by current theory. Before a predictive theory has been developed, there is even some censorship or bias risk in using neuroscientific terms.

[lutusp]

>> explain in neuroscientific terms

> Isn't that putting the cart before the horse?

Not in science. In science, the explanation is both the cart and the horse. No explanation, no science. Einstein didn't win a Nobel Prize for noting that electrons are emitted by a metal surface, he won for explaining why they are emitted. Had Einstein been a psychologist, publishing the fact that electrons are emitted (for simply describing) would have been enough.

> The article draws attention to a phenomenon that isn't predicted by current theory.

That's uncontroversial, since there are no theories in psychology, only descriptions. This, by the way, is why the director of the NIMH recently decided to abandon the DSM, to so-called "bible" of psychiatry and psychology, on the ground that it only contains descriptions and therefore has no scientific value (the DSM will remain as a diagnostic guide):

http://www.nimh.nih.gov/about/director/2013/transforming-dia...

Quote: "... symptom-based diagnosis, once common in other areas of medicine, has been largely replaced in the past half century as we have understood that symptoms alone rarely indicate the best choice of treatment ... Patients with mental disorders deserve better."

> Before a predictive theory has been developed, there is even some censorship or bias risk in using neuroscientific terms.

I think there's little risk in asking "Where's the science?"

[pradocchia]

In science, the explanation is both the cart and the horse.

I'm sorry, but no. Observation comes first, then hypotheses, then prediction, then verification. Explanations go from proposed to confirmed, but they are certainly not the genesis of scientific knowledge. The phenomenon itself must come first, otherwise all you have is the fitting of facts to theory.

So, we have observations w/out a coherent, compelling, or generally agreed-to theory. If and when a successful theory is developed, it will predict observations to date and predict more effects not yet observed or observed and ignored. Well, that sounds like a pretty exciting field of science, actually.

[lutusp]

>> In science, the explanation is both the cart and the horse.

> I'm sorry, but no. Observation comes first, then hypotheses, then prediction, then verification.

The corpus of scientific theory is a set of tested, falsifiable explanations. Legitimate sciences don't rely on mere descriptions, even well-tested ones.

But let's take your claim and test it scientifically -- let's assume that we don't need explanations, we can get by with your stated criteria: observations, "hypotheses, then prediction, then verification." Here goes:

Let's say I'm a doctor and I've created a revolutionary cure for the common cold. My cure is to shake a dried gourd over the cold sufferer until he gets better. The cure might take a week, but it always works. My method is repeatable and perfectly reliable, and I've published my cure in a refereed scientific journal (there are now any number of phony refereed scientific journals). And, because (in this thought experiment) science can get along without defining theories, I'm under no obligation to try to explain my cure, or consider alternative explanations for my breakthrough — I only have to describe it, just like a psychologist.

Because I've cured the common cold, and because I've met all the requirements that psychology recognizes for science, I deserve a Nobel Prize. Yes or no?

Ask yourself what's wrong with this picture, and notice that the same thing is wrong with psychology — all description, no explanation, no established principles on which different psychologists agree, no effort to build consensus, and no unifying theories.

> So, we have observations w/out a coherent, compelling, or generally agreed-to theory. If and when a successful theory is developed, it will predict observations to date and predict more effects not yet observed or observed and ignored. Well, that sounds like a pretty exciting field of science, actually.

Yes -- and shaking a dried gourd can cure the common cold.

[pradocchia]

To continue your thought experiment, another researcher designs a new, double-blind trial of gourds, and discovers that while gourds per se have no effect, the placebo effect is statistically significant.

There is no explanation for WHY expectations would effect outcomes. In fact, any explanation at this stage would be provisional and highly suspect. And yet, there it is. What are we to do, ignore this phenomenon as if it didn't exist? Question the statistical ability of the researcher, and all other researchers who document a placebo effect?

And lo, the messenger. Ready, aim, fire.

[lutusp]

> To continue your thought experiment, another researcher designs a new, double-blind trial of gourds, and discovers that while gourds per se have no effect, the placebo effect is statistically significant.

Without a search for causes, for explanations, even the placebo effect is routinely disregarded. For example, it has been recently discovered that all psychological therapies are equally efficacious. Until now, the assumption was that Cognitive-Behavioral Therapy was superior to others, but that's been disproven. But, even though all therapies produce the same outcome, no one in psychology seems willing to consider the idea that it's all placebo effect.

> In fact, any explanation at this stage would be provisional and highly suspect.

Except the one that Occam's razor suggests, the default assumption under these circumstances: placebo effect. Or, perhaps better, the non-explanation suggested by the null hypothesis -- nothing meaningful has been measured and no conclusions can be drawn, which I think is your point.

> What are we to do, ignore this phenomenon as if it didn't exist?

No, but as scientists, we would do well to avoid drawing any conclusions not supported by rigorous experiment -- including the responsibility to propose and then test a theory about what's been observed.

[md224]

I'm not really sure what you guys are arguing about... obviously observations and descriptions are an important first step toward developing scientific theories. Lutusp seems to be complaining about people who feel descriptions are sufficient and make no effort to discern the underlying mechanism. Is there really disagreement on this point?

If there is, then maybe this is just a debate about Instrumentalism:

http://en.wikipedia.org/wiki/Instrumentalism

[lutusp]

> Lutusp seems to be complaining about people who feel descriptions are sufficient and make no effort to discern the underlying mechanism. Is there really disagreement on this point?

Yes, among psychologists, who insist that explanations aren't necessary, that it's science even if no one tries to identify a cause for the effect being measured. But this assumption is now under serious challenge, as more and more emphasis is being placed on a search for causes, to the degree that the director of the NIMH recently ruled that the DSM (psychology's "bible") will no longer be accepted as a source for science (it will remain as a diagnostic guide).

The practical meaning of this change is that researchers who apply for funding through the NIMH will need to avoid using the DSM's symptomatic categories as a basis for research -- they instead need to express their proposals in more scientific terms, in terms of causes, not just effects. In other words, explanations, not just descriptions.

[arkades]

"I'm sorry, but no. Observation comes first, then hypotheses, then prediction, then verification. Explanations go from proposed to confirmed, but they are certainly not the genesis of scientific knowledge.

Not really. I mean, I appreciate the old "Scientific Method" card, that doesn't actually describe how it happens, pretty much ever. All observations happen schematically: by the time you're in grade-school, you've already gotten a basic grounder in Science: The Lies to Children Edition, and all future observations and learning are elaborations on the groundwork. Observations without theory happen at somewhere around, I don't know, age 2. By the time you can speak, you've already started the process of sense-making. In reality, observation always happens against the background of pre-existing theory. It's turtles, all the way down. (This is hardly ground-breaking: Thomas Kuhn was discussing this half a century ago).

"The phenomenon itself must come first, otherwise all you have is the fitting of facts to theory."

Which is precisely what we do. We collect facts, interpret them (that is, use them as a representation of underlying trends or relationships), and then collect more facts to see if our generalization holds true beyond the initial dataset that we used to generate our ideas. Science is absolutely about fitting facts to theory, and then collecting more facts to test that theory, and then elaborating that theory based on those new facts. It's entirely circular (for at least the last few centuries): no modern scientist sets out gathering observations without any pre-existing theory in his head.

[pradocchia]

Right, so you are describing what Kuhn termed the period of "normal science": theory in hand, collect new facts, fit facts to theory, repeat.

HOWEVER, that theory-in-hand rests on an earlier paradigm shift, where previously anomalous observations were re-integrated via some new conceptual framework that better accounted for all observations, not merely the conformed ones that scientists had focused on in the prior period of normal science.

So yeah, perhaps in the day-in, day-out existence of professional scientists, most work is fitting facts to theory. However, that theory exists because facts come first, because at some point in the past, mounting factual evidence overwhelmed the theoretical biases of an earlier generation.

Do I really need to spell this out for the HN crowd? For you, who reference Kuhn?

[lutusp]

> no modern scientist sets out gathering observations without any pre-existing theory in his head.

Or without the intent to shape a new theory, a new, testable inductive generalization, as when Einstein shaped special relativity. That's my favorite example because it happened in a theory vacuum, no pun intended. There must be a theory to inform the research, or the research must lead to a testable theory, or both.

[jwr]

> Not in science. In science, the explanation is both the cart and the horse. No explanation, no science.

Fair enough. But that doesn't mean that whenever you encounter the boundaries of science you get to call people derisive names and ridicule them.

See my other comment — I used to have the same smug approach, until I found out on myself that there are things that medicine doesn't understand. Of course I don't call it "science", but I don't ridicule it, either. I just know that we don't know.

[lutusp]

> But that doesn't mean that whenever you encounter the boundaries of science you get to call people derisive names and ridicule them.

Not an issue except to the oversensitive. Consider the words of NIMH director Insel as he recently announced that the DSM (psychiatry and psychology's "bible") is to be abandoned:

http://www.nimh.nih.gov/about/director/2013/transforming-dia...

Quote: "The strength of each of the editions of DSM has been “reliability” – each edition has ensured that clinicians use the same terms in the same ways. The weakness is its lack of validity ... Patients with mental disorders deserve better."

To a psychiatrist or psychologist, in particular those responsible for the content of the DSM, calling it invalid and unscientific might be taken as personal criticism. But that's unavoidable if we're going to move away from unscientific practices.

> See my other comment — I used to have the same smug approach, until I found out on myself that there are things that medicine doesn't understand.

That isn't a legitimate comparison. In medicine, if something isn't understood, a researcher says, "We don't understand this." In psychiatry and psychology, the response is, "Take some Zyprexa and call me in the morning."

Personally, I look forward to the day when "NOS: Not Otherwise Specified" will no longer be looked on as a legitimate diagnosis meriting treatment.

[Locke1689]

He's not talking about medicine, he's taking about psychology. I'm sure molecular pathology gets a very favourable "is science" score.

[kybernetikos]

There are a number of people through the ages who have greatly advanced science by providing accurate observations. In fact, many scientists specialize in working out how to test the explanations of others.

It seems somewhat odd to say that Tycho Brahe was not doing science when he compiled unprecedentedly accurate astronomical tables that were the foundation for revolutions in the way we understand the universe, or to say that Arthur Eddington was not doing science when he measured the deflection of light during a solar eclipse.

The first (and many subsequent) Nobel prize in physics was given for a discovery, not an explanation.

Rutherford famously said "All science is either physics or stamp collecting", and I'm sure he meant to denigrate the stamp collecting aspect, but the fact is that it's absolutely key to science.

[lutusp]

> There are a number of people through the ages who have greatly advanced science by providing accurate observations.

We need to distinguish between scientific observations and scientific theories. A scientific theory, in particular the sort that ends up defining scientific fields, must be in the form of a tested, falsifiable explanation of observations, preferably one that predicts new observations yet to be made.

Psychology's problems don't result from an absence of scientists -- there are plenty. But until those scientists start crafting falsifiable theories about their observations, psychology will remain a pseudoscience.

The peculiar thing is that psychologists don't realize this. It has something to do with how they're trained. At some point, they're told that, with respect to the mind, looking for causes is a fool's errand, and they assume they can create science without proposing and testing explanations for what they observe.

But they can't -- science doesn't work that way. As a result, psychology has produced any number of dried-gourd cures over the years.

[AdrianRossouw]

his point is that these studies never seem to get to that point. they never even seem get to the point of forming a predictive theory.

'seem' is the operative word here.

[coldtea]

>Ahh. Psychologists. Expectations calibrated.

A, a positivist reductionist. Expectations on intellectual sophistication and ability to comprehend the world beyond simplistic models calibrated.

[lutusp]

>> Ahh. Psychologists. Expectations calibrated.

> A, a positivist reductionist.

With respect to psychology, given its history, the burden isn't on me to avoid faulty generalizations, it's on psychology to overcome the weight of its past.

> Expectations on intellectual sophistication and ability to comprehend the world beyond simplistic models calibrated.

At a time when the director of the NIMH has decided to abandon the DSM on the ground that it's not scientific enough to take seriously?

http://www.nimh.nih.gov/about/director/2013/transforming-dia...

Quote: "The strength of each of the editions of DSM has been “reliability” – each edition has ensured that clinicians use the same terms in the same ways. The weakness is its lack of validity ... Patients with mental disorders deserve better."

What appears to be a simplistic generalization is, in this case, the result of much reflection and analysis, and a reluctant but legitimate conclusion.

[jasonkolb]

I think the phrase you're looking for is "ad hominem".

[coldtea]

I don't think "ad scientum field" is any better than an "ad hominem".

Not to mention that mine was not an ad hominem in the first place. I attacked an epistemology/methodology "positivism/reductionism" that he exhibited (as far as I can tell), not his person.

[nixy]

> "Expecting to know the answers made people more likely to get the answers right."

Well, yes, if someone expects you to know the answer it seems pretty straight forward that you would be more inclined to make a guess rather than saying "I don't know." All of us who ever watched a quiz show on TV know that people who guess an answer instead of passing will sometimes get it right. More often than people who pass, actually.

[coldtea]

>Well, yes, if someone expects you to know the answer it seems pretty straight forward that you would be more inclined to make a guess rather than saying "I don't know."

And what makes you think that they didn't account for the "I don't knows"?

It generally surprises me how, from reading an 100 high mile description of an article, some commenters always assume (without any evidence) that the results are due to some extremely naive methodological mistake.

[vidarh]

But did they account for the "I don't know so I'll just randomly pick something" vs the "I should know this, so I'll reason about it" groups?

If you're teaching a subject to someone, you will often come across situations where what is lacking is not knowledge, but confidence or willingness to apply the knowledge. I see that often in my son, and I've seen that at work: People say they don't know, or make a crap guess even in situations where I know that they know the actual answer. Often some prodding or "confidence boosting" will make them produce the real answer very quickly, and often subsequent answers appears to be forthcoming a lot quicker.

It'd be very surprising to me if you can't systematically improve peoples response by increasing their confidence in their ability to answer. The more interesting question to me is by how much, and with how little encouragement.

[Panoramix]

I think it's due to expectations. Every single time I read about psychology research there are these glaring gaps in methodology that completely nullify their experiments. After losing track of how many psych papers have used statistics wrongly, planned their experiment poorly or had seriously questionable methodology I have lost faith in this field. It's too much work going through a paper every time something reaches the news.

I therefore make the very personal and questionable choice to ignore (most of) them and go straight to the neuroscience people - which in general have a way higher understanding and higher scientific standards.

[lutusp]

> ... some commenters always assume (without any evidence) that the results are due to some extremely naive methodological mistake.

Since the article doesn't presume to explain its results, the presence or absence of mistakes is moot. Science isn't about descriptions -- that's metrology. The threshold of science is crossed when someone dares to offer a testable, falsifiable explanation. But in psychology, that rarely happens, and psychology has no central defining theories (explanations) such as are found in scientific fields.

[Bootvis]

It at least explains why I'm so smart ;)

[AdrianRossouw]

you dare question a website with 'scientific' in the URL? I like your moxie.

I recall reading something posted recently about how the way the grants in social sciences is set up ensures that you get a whole lot of papers that have a catchy 'truthy' hook to them, which will then get disseminated by the mainstream media and quickly forgotten.

It basically turned the fields into little more than generators of soundbites.

[lutusp]

> you dare question a website with 'scientific' in the URL? I like your moxie.

Yep, I had a lot of nerve doing that. :)

A while ago, in an article I noted that Wikipedia defined neuroscience as "the scientific study of the brain and nervous system", while psychology was defined as "the study of the mind, partly through the study of behavior." Within hours of my article's appearance someone inserted the word "scientific" into psychology's definition. Solved that problem.

> I recall reading something posted recently about how the way the grants in social sciences is set up ensures that you get a whole lot of papers that have a catchy 'truthy' hook to them, which will then get disseminated by the mainstream media and quickly forgotten.

Yes, even to the extent that two studies arrive at opposite conclusions but don't notice each other (and no one points out the contradiction). My favorite example of overlooked contradictions are two current, well-regarded psychological theories -- Grit and Asperger Syndrome.

http://en.wikipedia.org/wiki/Grit_(personality_trait) : "Grit in psychology is a positive, non-cognitive trait, based on an individual’s passion for a particular long-term goal or endstate coupled with a powerful motivation to achieve their respective objective."

So according to the Grit contingent, focusing on a few activities, or just one, is a "good thing", and typical of successful people.

http://en.wikipedia.org/wiki/Asperger_syndrome : "Asperger syndrome (AS), also known as Asperger's syndrome or Asperger disorder, is an autism spectrum disorder (ASD) that is characterized by significant difficulties in social interaction, alongside restricted and repetitive patterns of behavior and interests."

So according to the Asperger contingent, focusing on a few activities, or just one, is a mental illness.

How can this happen? The answer is that there's no central defining theory in psychology, so people are free to draw conclusions that don't need to be compared to tested, defining principles like relativity or evolution.

> It basically turned the fields into little more than generators of soundbites.

Well put. :)

[vidarh]

> So according to the Asperger contingent, focusing on a few activities, or just one, is a mental illness.

You chose to ignore the requirement of "significant difficulties in social interaction". These are also not the only diagnostic criteria.

When you make comments like this, it becomes very hard to take your other criticisms of psychology seriously, as it raises the question as to what else you're ignoring/leaving out or unaware of.

The entire point of a syndrome is that it is a grouping based on the presence of a certain set of required symptoms combined by some threshold of additional symptoms, rather than a clearly identifiable underlying cause.

Many individual symptoms of a syndrome can be perfectly normal on their own or even strong desirable at certain degrees, but become substantial problems for the individuals involved when combined together, or when the trait is too intense. There's in other words no contradiction between the positive treatment of grit vs. the description of Asperger syndrome as a disorder.

[lutusp]

>> So according to the Asperger contingent, focusing on a few activities, or just one, is a mental illness.

> You chose to ignore the requirement of "significant difficulties in social interaction". These are also not the only diagnostic criteria.

But much field experience shows that the absence of the full symptom set doesn't prevent the diagnosis, therefore the comparison is valid. During its tenure as a serious syndrome Asperger's diagnoses went completely out of control, due to ambiguous diagnostic criteria and (my point) how much like normal behavior the symptoms are.

Allen Frances, the editor of DSM-IV (the edition that introduced Asperger's), now thinks the inclusion of Asperger's was a mistake that led to what he now describes as a "phony epidemic" of diagnoses. Over the protests of many psychologists, Asperger's has been removed from the new DSM.

> There's in other words no contradiction between the positive treatment of grit vs. the description of Asperger syndrome as a disorder.

False. The contradiction is obvious -- they both describe the same behavior. And when it was pointed out that many very successful people exhibited Asperger's symptoms (or "Grit" symptoms, depending on one's outlook), psychologists responded by labeling those people mentally ill. As a result, Isaac Newton, Albert Einstein, Thomas Jefferson, and Bill Gates have been labeled mentally ill. The evidence? They accomplished something noteworthy by focusing their attention on a few activities, or just one.

But you're missing the point, which is that psychology proceeds based on descriptions, not explanations. Asperger's is a description without an explanation. "Grit" is a description without an explanation. Science requires explanations. This is why the DSM, psychiatry and psychology's central authority, is in the process of being abandoned as unscientific.

> There's in other words no contradiction between the positive treatment of grit vs. the description of Asperger syndrome as a disorder.

That's true, and it shows what's wrong with psychology -- its superficiality and willingness to accept mere descriptions in lieu of explanations.

[floobynewb]

This is a real thing. There appear to be intimate connections between the mind and the body, in both directions. Levels higher up the hierarchy of dynamical systems within an organism appear to direct the function of lower systems. It's feedback loops all the way down.

[lutusp]

> There appear to be intimate connections between the mind and the body, in both directions.

That may be true, but only one of those can be studied scientifically.

[floobynewb]

I'm not sure I follow. There's nothing mystical about the idea. The hypothesis is testable as shown by the research. The data currently looks to support it. I'm not sure why you are saying it is unscientific. I grant you there may be alternative explanations for the data, but it is often true that science has been lead astray by incorrect framing of the question, but it's still science...

[lutusp]

> I'm not sure I follow.

The mind is not -- cannot be -- a source of empirical evidence. Science requires empirical evidence.

> The hypothesis is testable as shown by the research.

What research is that? Which scientific theory did the research either shape, or test, or potentially falsify?

> I'm not sure why you are saying it is unscientific.

The article describes, it doesn't explain. Science requires explanations, explanations that can be generalized into principles, tested, and potentially falsified.

> I grant you there may be alternative explanations for the data ...

But the data aren't explained, they're described. Test subjects were more likely to answer correctly if they received positive encouragement. That's a description. No one has tried to explain or generalize that description, turn it into something falsifiable.

> it is often true that science has been lead astray by incorrect framing of the question,

Absolutely true.

> but it's still science...

Absolutely false.

[randallsquared]

> The mind is not -- cannot be -- a source of empirical evidence. Science requires empirical evidence.

Since that's clearly not true of the brain's behavior, you seem to be making a claim that the mind is supernatural. Is that your position?

[lutusp]

>> The mind is not -- cannot be -- a source of empirical evidence. Science requires empirical evidence.

> Since that's clearly not true of the brain's behavior, you seem to be making a claim that the mind is supernatural.

No, only that the mind is not a legitimate source of empirical evidence (a negative assertion). Consider the areas that science doesn't (cannot) cover, and notice what they have in common -- usually, the inability to gather empirical evidence on whose meaning similarly equipped observers can agree.

For me to say that the mind cannot produce empirical evidence is uncontroversial -- it's certainly true. For me to claim that the mind is a supernatural entity, I would need to produce evidence for that conclusion. But I can't, so that assumption is itself unscientific.

[zipdog]

It would be interesting to know what the questions were (or even what genre) - to get a better sense of the significance of getting them right.

Unfortunately there is no indication in this article, or the abstract of the paper, and the journal article itself requires a subscription.

[guelo]

The article includes a bunch of references to placebo effect research by neuroscientists.