[AnthonyMouse]

People always say they have data. Then you look at the data and it's full of confounders or confuses cause and effect or expects to see a stark instantaneous effect from something that gradually affects long-term population-level life choices.

Most data is trash. If the data is contrary to reason then either the reasoning is wrong or the data is wrong, but for anything with public policy implications it's more likely to be the data because policy data is disproportionately generated by people trying to influence the result.

Also notice that the benefits of the policy aren't limited to the thing you're disputing. Eliminating transit fares still removes a regressive tax, increases the government's revenue collecting efficiency by eliminating the transit collections infrastructure costs and improves privacy by stamping out a mass surveillance system regardless of what extent it reduces traffic congestion. Which means that if you're trying to reduce traffic congestion, you should definitely start there because it very well could work and it's something you should be doing anyway.

[stouset]

> If the data is contrary to reason

Literally everything in behavioral economics is contrary to reason because people simply do not behave like rational utilitarians in the real world, either individually or as a collective.

We are a species where people routinely spend ten minutes circling around to find a parking spot that's thirty seconds closer to their destination.

[AnthonyMouse]

> Literally everything in behavioral economics is contrary to reason because people simply do not behave like rational utilitarians in the real world, either individually or as a collective.

People largely do behave rationally, it's just that rational behavior includes things like heuristics to account for incomplete information or trading the optimality of the choice against decision time costs.

This is one of the reasons the data is always such a mess. You make a change and people don't immediately notice because they're still applying their old heuristics or haven't recognized that the new alternative is available yet. Then the data shows nothing relevant. Meanwhile five or ten years later people have largely figured it out, but by then a dozen other things have also changed and there is no way to measure the result of the original change net of the others whose true effects are also unknown.

This is why actual science uses double blind randomized controlled trials, but this happens for policy data approximately never.

[stouset]

We can also compare locales that implement do implement $policy and ones that don't. As with anything it's not perfect but it can be indicative.

For the exact same reason it's often so difficult to tease out the impact of policies in data, it's even harder to reason through what will happen from first principles. And yet for someone who wants to hold up actual science and double-blind studies, you seem awfully eager to throw out what precious little data we do have to just go with your gut.

That's not how science works.

[AnthonyMouse]

> We can also compare locales that implement do implement $policy and ones that don't. As with anything it's not perfect but it can be indicative.

It's so imperfect it often inverts the sign of the result, e.g. cities that implement some anti-X policy have even more X than cities that don't, but the reason is that the cities that implemented the policy were the ones expecting the biggest problem. Then a policy that was effective in reducing the growth rate of X from 30% to 10% is accused of increasing X by 10%.

> For the exact same reason it's often so difficult to tease out the impact of policies in data, it's even harder to reason through what will happen from first principles.

The point isn't that you can't get the reasoning wrong, it's that most of the data is just mud. It's knowing that X = Y + Z and then someone gives you the value of Y, which is data, and asks you to solve for X. You still have no ability to answer the question.

But okay, fair enough, you have some data, you're maybe one step closer to solving the problem (even though you haven't yet). Here's what gets me. If the data contradicts the expectation, you should now have a theory for why. Is it because the data has confounders and hasn't really demonstrated anything one way or the other? Is it because you made mass transit free but it still didn't cover the path to where people live, and in order to be effective you have to do both? What's the reason? That is an important question because it determines what you should do instead, if anything.

Just "data says that doesn't work" isn't really refuting anything. If the reason is confounders then the data isn't even a counterargument. If the reason is that it doesn't work as implemented there but there is a variant that works, that's important, but then you can still do it by adopting the variant. And if it actually doesn't work at all then where's the explanation? How do you know what to do instead?

For example, if people don't respond to price differences then congestion pricing shouldn't work either, right? If price differences do work but part of the price has to be an increase to trigger loss aversion and then it's the total price that matters, what if you make mass transit free and then increase the federal gas tax instead of using congestion pricing? People see the higher price, compare it to the lower price and you get a double effect but still don't need to pay for separate collections infrastructure or have the privacy cost and the gas tax also encourages fuel efficiency.

But if the data is just some convoluted mud that can't answer the questions then it's worse than nothing, because it makes you think you know something you don't actually know.