[Ozzie_osman]

Data is not a substitute for good judgment, for empathy, for proper incentives.

The article focuses on governments and bureaucracies but there's no better example than "data-driven" tech companies, as we A/B test our key engagement metrics all the way to soulless products (with, of course, a little machine learning thrown in to juice the metrics).

I wrote about this before: https://somehowmanage.com/2020/08/23/data-is-not-a-substitut...

[mminer237]

I think it's almost worse in tech because it largely works. If the government sets a flawed metric, their real goal of pleasing their constituents has failed and theoretically they either have to fix it or lose political support.

But in tech, if your goal is just to make money, soullessly following data will often get you there, to the detriment of everyone else. Clickbait headlines will get you more views. Full-page popup ads will get you more ad clicks/newsletter subscriptions. Microtransactions will get you more sales. Gambling mechanics will get you more microtransactions.

You can say it's a flawed metric, but I think in the end, most people just actually care more about making money than they do about building a good product.

[gen220]

I've written the same sentence before! This is so cool! pardon the wall of text.

Here's my thesis, curious to hear your thoughts.

At some time around 2005, when efficient persistence and computation became cheap enough that any old f500-corp could afford to endlessly collect data forever, something happened.

Before 2005, if a company needed to make a big corporate decision, there was some data involved in making the decision, but it was obviously riddled with imperfections and aggregations biases.

Before 2005, executives needed to be seasoned by experience, to develop this thing we call "Good Judgement", that allows them to make productive inferences from a paucity of data. The Corporate Hierarchy was a contest for who could make the best inferences.

Post-2005, data collection is ubiquitous. Individuals and companies realized that you don't need to pay people with experience any more, you can simply collect better data, and outsource decision-making to interpretations of this data. The corporate hierarchy now is all about how can gather the "best" data, where "best" means grow the money pile by X% this quarter.

"Good Judgement" used to be expected from the CEO, down to at least 1-3 levels of middle management above the front-line people. Now, it appears (to me) to be mostly a feature of the C-Suite and Boards, and it's disappeared elsewhere. Long-term, high-performing companies seem to have a more diffused sense of good judgement. But these are rare. maybe they always have been?

Anyways, as we agree, this has a tendency to lead in problematic directions. Here's my thesis on "why".

Fundamentally, any "data" is reductive of human experience. It's like a photograph that captures a picture by excluding the rest of the world.

Few people seem to understand this analogy, because they think photographs are the ultimate record of an event. Lawyers understand this analogy. With the right framing, angle, lighting (and of course, with photoshop), you can make a photograph tell any story you want.

It's the same issue with data, arguably worse since we don't have a set of standard statistics. We have no GAAP-equivalent for data science (yet?).

Our predecessors understood that data was unreliable, and compensated for this fact by selecting for "Good Judgement". The modern mega-corps demonstrate that we don't have a good understanding of this today, evidenced by religious "data-driven" doctrine, as you describe.

People will say "hey! at least some data is better than no data!", to which I'll say data is useless and even harmful in lieu of capable interpreters. In 2021, have an abundance of data, but a paucity of people who are capable of critical interpretation thereof.

I don't know if it's a worse situation than we had 20 years ago. But it's definitely a different situation, that requires a new approach. I think people are taking notice of it, so I'm hopeful.

[cloogshicer]

Thank you for writing this, I enjoyed reading it and largely agree.

[texasbigdata]

Same, but the sentance "Individuals and companies realized that you don't need to pay people with experience any more, you can simply collect better data, and outsource decision-making to interpretations of this data." is probably demonstrably false and at a minimum it'd be very difficult to prove its true over a bull hypothesis that were as good as, if not marginally better than before as a species.

To an extreme it's "well we got horse carriages and cars so like no one is ever gonna run a fast marathon ever again, were outsourcing everything to the hay eaters".

Yeah no. And not that we shouldn't perhaps be more contemplative as a society, but as a species we generally don't atrophy capability that would other wise be complentary and beneficial to us.

Case and point, it's relatively well known "management ability" is a thing across cultures, experience, industry and training. The metrics point to at least in a narrow sense data driven executives outperforming their "solely intuitive" bunch.