[hej]

You could write the same about pretty much any popular theory.

Looking at just the first few newspaper articles and surveys was enough to conclude that the media agenda influences the public agenda. After that, it was just a question of watching the evidence pile up.

It seems like she went into this without a hypothesis, a (relatively) superficial look at the paintings let her form that hypothesis, and further, more in depth looks at the paintings only confirmed and didn’t contradict that hypothesis.

That seems like the normal and appropriate process for discovering things like this.

Yes, confirmation bias is a problem, but it always is. That’s just how humans work. The mere existence of confirmation bias (which makes it plausible that confirmation bias was at work here) doesn’t immediately discredit the results.

I’m trying to say that confirmation bias might well be a problem here (though the presented evidence seems strong and I personally can’t think of good alternative explanations for what she found – but I’m also not an art researcher) but that is nearly always the case (when it comes to squishy things like this, I’m not necessarily talking about physics, but it’s certainly also a possibility there). So yeah, you can write “Confirmation bias anyone?” under a great number of scientific papers, but that alone is not very valuable and constructive.

Sometimes the process of discovery itself just cannot be sufficiently protected from confirmation bias (in the sense of excluding any chance of confirmation bias), especially when it comes to something like looking at a painting and trying to figure out its history. Then you have to just roll with it while also being aware that it’s a huge problem (preferably by going through your evidence with a fine tooth comb, always looking for alternative explanations – better yet, you should get many other people to do that, and I bet they will in this case).

[xanados]

The logical conclusion of your argument isn't that this theory should be more credible, but that we should assign all such theories a lower likelihood of representing reality. Luckily most of these theories have no bearing on how anyone lives their lives so it hasn't impeded human progress too much.

[hej]

That is indeed the logical consequence. I’m not disagreeing.

There are a plethora of factors that make, say, results of a social science statistical analysis less likely than they might seem. Biases, wrong assumptions during statistical analysis, …

I’m also not claiming that it’s impossible to find better methods that shield scientists much better against, for example, biases. I do indeed think that this (developing new and better methods) is where non-natural sciences have their most important work in front of them. This is where it’s at for them, this is where real revolutions of several fields are possible. If only someone knew how exactly.

Until then we have to deal with what we have. I think it works relatively well, considering, but I’m certainly not happy about the current state.

When it comes to this particular theory, it seems to me that the odds are pretty good that it is true. The explanation for why the copy is the way it is – it was painted by someone with a knowledge of how the original was painted – doesn’t seem that outrageous and pretty logical to me. It’s certainly always possible that there is a better one.

(By the way, I disagree vehemently that those kinds of theories have little impact on our lives. This one maybe not, but forensic science – which is very similar in its method and process to what this researcher did – impacts a great many people. There researchers will often take a first look at something and immediately start forming hypotheses, which they then seek to confirm or falsify by further collecting evidence. Even worse: Researchers might be told hypotheses by non-researchers – say the police. It’s important that we find better ways to do it. It's important that we know exactly how sure or unsure we can be about the results.)

Again, to re-iterate: Just because there is justified doubt doesn’t mean this research is worthless (which your tone seemed to imply, maybe I’m wrong about that).