Economic Junk Science
By Arnold Kling
I am reading the latest issue of the American Economic Review, which is a selection of papers from this year’s annual convention of the American Economic Association. A lot of it is what I consider to be “junk science.” So far, no happiness research, but plenty of stuff that is just as bad.If you want to see what I consider to be junk science, look at the paper that Bryan just praised. It says that differences in health are caused by differences in IQ.
I think that there is a good common-sense argument that intelligence would affect health. As the article notes, if you think about
life’s daily demands for continual learning, spotting of problems,
and reasoning, especially in health self-care,
it is easy to see where intelligence would play a role.
The junk-science aspect is to look at correlations and say that they prove what intuition and common sense might wish to be true. I just do not believe that it is possible to use correlation to untangle the relationships among health, socioeconomic status, and intelligence.
One of my pet issues is measurement error. My guess is that IQ measures something very accurately. Leave aside what it measures, exactly, because I am not sure about that. But give the same group of people IQ tests many years apart, and the correlation is remarkable. I don’t think you can say the same thing about personality tests, such as Myers-Briggs, or about measures of socieoeconomic status.
Because IQ is such a good measure (again, being agnostic about what it measures), it tends to correlate really well with other things. That is just the algebra of correlation. If, say, 80 percent of the variation in what I claim is a measure of socioeconomic status is pure noise, then it will not correlate well with anything. Meanwhile, if the measurement-error component of IQ is low, it will drive my hypothetical SES variable out of any equation, regardless of the true underlying causal model.
In reading the American Economic Review, I came across this paper by Avner Greif, an exponent of the New Institutional Economics. It contained no statistical correlations, only assertions backed by common sense. I imagine that, to many economists, Greif’s article will seem methodologically inferior.
I can see where Greif’s article would be frustrating. He says that corporations emerge to solve problems of social capital formation in societies where nuclear families replace extended tribal families as the key units. There are all sorts of issues in sorting out correlation and causality in that model, and rather than pulling any statistical tricks out of a hat, he mostly just slides past such issues.
But my view is that the statistical tricks are just that–tricks. That is why I don’t care much for Steve Levitt’s work. I think that in the end we have to rely on common sense. And where two very different common-sense analyses are plausible, we have to just accept that we do not have a firm basis for choosing one over the other.
Using correlations as substitutes for controlled experiments simply gives a false sense of security to one’s confirmatory bias. It is junk science, not advancement of knowledge.