Hacker News new | ask | show | jobs
by jrochkind1 4406 days ago
In this age of metrics everywhere, evidence-based everything, big data...

I think what this story actually is, is a reminder that just because something is based on quantitative data, that doesn't make it 'objective' or automatically 'truth'.

Quantitative analysis rests on so many choices, as to how to measure, what measurements to use, what statistical formula to use, how to interpret what they say. Each of those choices can be mistaken -- _or_, even more troubling to the worldview that quantitative==objective truth, be subject to debate among reasonable and well-intentioned people about the best choices to make and the implications thereof.

I'm not saying it means there is no 'truth', and all research conclusions are equally valid. I'm saying that research conclusions based on quantitative data, no less than those based on qualitative information, are subject to debate and argument, not physical objective material reality simply because there were measurements and numbers involved.

5 comments

An excellent book on this topic is "A Social History of Truth: Civility and Science in Seventeenth-Century England." It examines what "truth" means in a scientific context by going back to its roots as a discipline.

The tl;dr version is that knowledge (even "hard" scientific knowledge) cannot be meaningfully acquired by a single person independent of a community that makes common assumptions, since going to very first principles for even the simplest analysis is totally unwieldy. Take something like simple genetics — we rely on the testimony of a lot of people just to accept the utility of a basic Punnett square.

I much preferred Summer's assessment of Piketty to FT's: he was measured and responsible in his criticism, and correctly noted that it'll take years for serious academics to sift through the merit of the book. Alas, responsible discussion is hard to come by when it comes to data that challenges peoples' closely-held values.

In this age of metrics everywhere and big data, the data on inequality, a very important social issue, took Piketty and his collaborators, all professional economists, many years of research to assemble and publish.

This data is not even big: to paraphrase a joke I read on HN recently, you could do the analysis in Excel and it wouldn't even crash.

There is something to be said for systematically gathering a lot more data than governments collect now, especially income and wealth data in this case. Not because it benefits anyone economically right now, but because without collecting it now, it is that much harder to reconstruct it later.

The primary issue here is how you can define "wealth" or "rich" and "poor." Income's one measure, sure. But what about the trust fund kids? What about college kids?

The real problem with collecting data like this is that there's no real good source of it. Most of what you get has selection bias, and the other stuff you can get is riddled with holes. And then you try to do it across countries, where definitions change and governments college disparate data at differing granularities and it just gets...frustrating.

An economist's toughest job is finding good data. The analysis is the easy part!

And even then most people don't have a choice--they have to use what data they are "stuck with" and often times that data is provided by people with their own agenda. Public-disclosure requirements is highly politicised and fought-over by various well-financed special interests. And that is not even counting the lawyers, accountants, and advisors paid to dress things up even further after a law or rule is created.
I'm interested to know where you think the debate should lie. Do you believe that it should be debated empirically among economic journals or in the court of public opinion? Because the book is a compilation of the former and we are only seeing doubt in the latter.
I like to say that usefulness trumps truth.
Usefulness to whom, and in what context? In the case of the climate change "debate", which has significant parallels to this one, denial is useful to entrenched interests at the expense of everyone else — including, ultimately, those same interests. How "useful" is that, exactly?
Thats the point. There is no truth only useful perspectives.