[jameslevy]

It's amazing to me how often Goodhart's Law can be observed in the wild, and I've come to believe it is an important enough concept that it should probably be something (along with logical fallacies and other concepts related to critical thinking) that should be prioritized above a lot of other things taught in high schools.

[novok]

So when you tell execs goodhart's law, they ask, ok what can we do about it when managing large organizations? Be it a 200 person department of a 30'000 person organization or even more?

[onos]

The usual argument is you need a metric and a check metric that can catch gaming the first. Eg unit sales and mean price per sale — the latter preventing a race to the bottom on price point.

[GauntletWizard]

You need at least a dozen to prevent gaming of the first. It's not even that hard, but you do need to be aware that you need many distinct metrics and you will bring at least one down at least somewhat, and hopefully by repeated iteration you will end up with everything better.

[novok]

And what prevents that when it becomes known there are n+1 metrics to game? And your doing this in a large organization, so it is statistically inevitable that people will want to game the metrics, bad actor or not?

[photonbeam]

Secret metrics

[lazide]

They pretty quickly stop being secret when they’re used, if it’s possible for them to be gathered and be a secret in the first place. :s

[bruce511]

The best approach is really hard. (Which is why proxy metrics in the first place.)

The best approach is to have a -very- clear set of goals, and to accurately measure progress to those goals, not to proxy metrics or goals.

Yeah, I know, easier said than done.

Usually a business has multiple goals, which are complementary. Enumerating, and measuring these make the system harder to game. Or more accurately make it harder to get better metrics without also getting closer to the actual goals.

If you want sales,then measure sales. If you want growth measure that. If you want customer satisfaction then measure that. If you want happy employees then measure that. List -everything-, measure everything. Understand the tensions, improvement in one area can drop the metrics in another. So work to keep improving them all.

All of this is -hard-. So most places don't bother. Find a simple measure (say stock price or turnover or profit) and ignore sustainability etc.

[BurningFrog]

If Goodhart's Law becomes commonly known, I expect some next-level fallacy will emerge in its place.