[crabmusket]

This reminds me a lot of the discussion of the scientific method by Karl Popper, and David Deutsch who was very influenced by Popper. "Being data-driven" sounds very empirical. Just look at the data, and see what you find in it.

But you can't just let the data "speak for itself" without an explanation or a theory that interprets the data. Popper in Conjectures and Refutations:

> Observation is always selective. It needs a chosen object, a definite task, an interest, a point of view, a problem. And its description presupposes a descriptive language ... which in its turn presupposes interests, points of view, and problems.

Deutsch, in The Beginning of Infinity, emphasizes the importance of conjecture, and the role of observation as refuting or criticising those conjectures:

> Where does [knowledge] come from? Empiricism said that we derive it from sensory experience. This is false. The real source of our theories is conjecture, and the real source of our knowledge is conjecture alternating with criticism. We create theories by rearranging, combining, altering and adding to existing ideas with the intention of improving upon them. The role of experiment and observation is to choose between existing theories, not to be the source of new ones. We interpret experiences through explanatory theories, but true explanations are not obvious.

To bring this back to the subject of the article, I might suggest that it's possible to be "data driven" without a sound explanation or theory that the data is either interpreted through, or used to criticise. Or maybe such theories do exist, but are left implicit.

[mikkergp]

Doesn’t the scientific method specifically say you can’t start with the data, you have to start with a hypothesis otherwise you are subject to all sorts of selection/hindsight biases. I mean you can start with data, but then you have to develop a hypothesis and use that to create an experiment that generates new data in order to reach a conclusion. It seems like that is the compromise the author is looking for, start with a good idea, then see if you can verify it with data.

[BeFlatXIII]

The scientific method as taught in K–12 schools is largely pablum. Often, the real process (beyond iterating off prior research) begins with collecting data, then noticing patterns to make a hypothesis to be tested with targeted data collection.

[musingsole]

A lot of this perspective depends on what point in time you choose as the start of the process. You can start with the hypothesis, or you can start with what gave rise to the hypothesis: exploration.

But, it's a layman's mistake to confuse the two and use it as a critique of the formalized scientific method.

Science bodies (like the NIH) explicitly forbid reuse or reinterpretation of data. An individual may use exploration as inspiration for a hypothesis...but for it grow into science out of curiosity requires new data generation from a carefully considered framework for the hypothesis.

[jpeloquin]

> Science bodies (like the NIH) explicitly forbid reuse or reinterpretation of data.

I think your main point is that collecting new data is necessary to test existing ideas. But reuse and reinterpretation of data is routine, e.g., in meta-analyses. It's not forbidden. You do have to disclose where the data came from.

[JackFr]

> But you can't just let the data "speak for itself" without an explanation or a theory that interprets the data.

If you look at the heart attack data, and you ignore smoking you end up inventing the mythical Type A personality — but it was data driven.

https://en.m.wikipedia.org/wiki/Type_A_and_Type_B_personalit...

[bluetomcat]

A single metric is just one very thin dimension from the temporal development of a complex process involving many factors. You need to watch a multitude of metrics to devise an explanative theory, and even then, that theory can be rendered flawed when new and unexpected factors come at play.

[tdehnel]

I think the point is the theory doesn’t come from the data (it can’t). It comes from the process of creative conjecture in a person’s mind.

[tshaddox]

The fact that empiricism is false was a revelation to me as a young adult, after reading so much about the triumphs of science and reason and getting very excited (mistakenly) that you can get away without bothering with pesky things like epistemology. Of course, Quine and others pointed out that empirical observations are useless without explanations both of the phenomenon being measured and the measurement device itself (including, for example, the human vision system). And I believe it was Deutschmark who pointed out that empiricism is itself an epistemology which had to be invented. It turns out that it tended to be a significant improvement upon previous widespread epistemologies, but that doesn’t mean it’s not false. :)

[js8]

I agree, pure empiricism can lead to superstition. If you only learn from experience, and do not have any theory that ensures the consistency of the model, it's easy to infer wrong causal connections.

[guerrilla]

> Observation is always selective. It needs a chosen object, a definite task, an interest, a point of view, a problem. And its description presupposes a descriptive language ... which in its turn presupposes interests, points of view, and problems.

Thanks, I'd never heard this quote before. He's pretty much describing pragmatism à la William James. I had no idea.

[crabmusket]

The pragmatists went a little bit too far in my opinion, though it has been a long time since I read any of them. Popper is describing observations, not reality.

I highly recommend Conjectures if you can find a copy. It's a short read and interesting.

[guerrilla]

What do you mean that they went too far? James and Peirce were not describing "reality" (in this discussion anyway. [1][2]) but rather were instrumentalists and thus saw every theory as having a purpose. That's the whole point of the squirrel argument. It not just "depends on what you mean" (as per analytic and some medieval philosophy) but also depends on what you're trying to do (which in turn depends on what you want/like.) In any case, the similarity I was pointing out is just that theories have purposes and ignoring this is a blatant blunder.

1. James even endorsed religion and other make-believe if it was useful to your purposes.

2. Peirce: "Consider the practical effects of the objects of your conception. Then, your conception of those effects is the whole of your conception of the object."

[crabmusket]

I think I'm out of my depth at this point actually, and maybe shouldn't have opined as readily as I did on the pragmatists. I did a little recap of where I'd encountered the pragmatists before and realised I only read Royce, who was a friend and interlocutor of James. But I don't think he could be called a member of the pragmatist school, so I shouldn't take the impressions I got from him to be representative!

The impression I had of pragmatism was that it made claims about absolute truth or reality. That's where I felt things were taken a little too far. But the impression I have may be a caricature or misunderstanding on my part.

Deutsch has a fair bit to say about instrumentalism in Beginning of Infinity which I will leave to the interested reader to discover.

[guerrilla]

Oh okay. I'm not an expert either but I think many might even consider them anti-realists but I'm not sure... I have a friend who is literally an expert, so I should probably ask and figure that out :)

> Deutsch has a fair bit to say about instrumentalism in Beginning of Infinity which I will leave to the interested reader to discover.

Okay, thanks, I'll check it out!

[ThomPete]

I was about to post that.

Here is a good talk https://www.youtube.com/watch?v=EVwjofV5TgU