By Arnold Kling
one of the problems with such research is the way that it’s presented to the public, through newspaper and magazine articles, or TED lectures, or the online dissemination of pre-pubication versions of papers. The stuff which gets the most attention has very rarely gone through the peer-review process, and as a result it’s often impossible for us, the final consumers, to get a good grip on what’s worth taking seriously and what’s just junk science.
As an aside, Salmon mentions the book by Raymond Fisman and Edward Miguel, called Economic Gangsters. I read the book a few weeks ago, and I have been meaning to blog about it. It has elements of Freakonomics in its description of the detective work of empirical analysis, and it has elements of The Undercover Economist in its focus on underdevelopment and its use of the basic economic model of incentives to explain behavior. I liked it a lot while I was reading it, but the longer it sat next to my computer waiting to be blogged about, the more it sort of faded for some reason. It’s way better than most books you are likely to be encouraged to read, but I’m not ready to give it a “must-read” recommendation.
As for Salmon’s larger point of when you should trust an economics paper, for me the key issue is how the authors treat potential objections. Basically, if the author ignores important potential objections, you cannot trust the paper. If the author meets the more important potential objections head-on, then that is better. As an amateur, your main disadvantage in evaluating a paper is perhaps not being as aware as a professional would be of potential objections.
I put no special value in peer review. You can have a peer who discards a paper for purely idiosyncratic reasons, or because he just does not understand key aspects of the paper. More often, you have a peer who approves a paper because it cites his own work favorably, which makes it immune to criticism in the eyes of the reviewer.