[quanto]

> A weak argument founded on poorly-interpreted data is not better than a well-reasoned argument founded on observation and theory.

So a good argument is founded on...good data and good understanding of data?

The article more seriously makes the mistake of begging the question: it presupposes the known classier of good and bad arguments and then goes on to say bad arguments with data is worse than good arguments. But how do you know good arguments from bad arguments in the first place? What makes a good argument if not empirical data?

[gwd]

> It presupposes the known classier of good and bad arguments and then goes on to say bad arguments with data is worse than good arguments.

It does indeed assume that there's a way to learn bad arguments from good; and so the focus should be on learning what are good argument and what are bad.

> ...What makes a good argument if not empirical data?

Consider the following conversation:

A: We've done some numbers, and we've determined that there's a correlation between the number of firemen at a fire and the total damage done by the fire; with the fires handled by a single crew of three firemen doing the least damage. So we should limit all fire responses to a single crew to minimize damage.

B: That doesn't make any sense -- of course we send more firemen to bigger fires, and bigger fires cause more destruction! If we take your advice, those big fires will cause even more damage!

A: Hey, my argument is backed by empirical data; yours is just theoretical!

Like, sure, it might be even better if B had empirical data to back him up; but even without that data, B should be winning the argument here. And the argument of the article is that many people espousing "data-driven" approaches end up being like A: Not scrutinizing the logic that they're using to analyze the data, and not acknowledging the limitations of what the data collected can say.

[jjk166]

You left out hypothesis C: if you send too many firefighters they get in eachothers way and become difficult to coordinate, making them worse at putting out the blaze.

And hypothesis D: fires with the same number of firefighters cause different levels of destruction because some departments are organized to let their 10X firefighters work more efficiently.

And hypothesis E: Many arsonists become firefighters thus more firefighters increases the risk that an arsonist will be on the team

And hypothesis F: The same as hypothesis A but since some tools require more than one person there's actually a minimum threshold below which destruction skyrockets

And hypothesis G: Wealthy areas that can hire more firefighters also suffer more expensive destruction for a given blaze.

And hypothesis H: If we invest the resources we're spending on firefighters into fire prevention we can reduce total fire damage

And infinitely more hypotheses.

There will always be another argument that makes some logical sense. And unfortunately reality is under no obligation to make sense, so it's entirely possible something that sounds stupid and counterintuitive could just happen to be correct anyways.

But with data, we can test hypotheses. Vary the number of firefighters and see what happens.

[tdehnel]

Good arguments (explanations) are hard to vary.

More here: https://www.lesswrong.com/posts/jcTsbaQ8hNc7qxwaQ/explanatio...