Paul Romer has recently been posting on the subject of intellectual honesty in economic research. I’m supportive of his argument that economists should behave like honest scientists, rather than lawyers who withhold information that is detrimental to their argument. I’m agnostic about his specific criticism of papers by people like Robert Lucas and Edward Prescott, but my hunch is that he’s at least partly right about the specific defects he finds. (I base this on nothing more than one of Lucas’s fans admitting that one of the papers did have one of the flaws claimed by Romer. I’m in way over my head, and no one should pay any attention to my views on the question.)

But I’d like to focus on where I disagree with Romer.

1. I believe intellectual dishonesty is very widespread in economics, and indeed in other sciences as well. Not complete dishonesty, as when people completely make up results. That’s pretty rare (I hope.) But Romer is trying to hold economists to higher standards than that, not just avoidance of outright lies, but also refraining from misleading claims.

Here’s just one example of why I think Romer is wrong in claiming that intellectual dishonesty is mostly confined to a small group. My experience is that empirical studies in economics are frequently tweaked until the researcher finds a “statistically significant” t-statistic of 2.0 or more. I can’t prove this, but I strongly believe it to be true. And yet I rarely see published papers that say “When I first did the study I found a t-statistic of 1.2, and then I tweaked the model over and over, and/or adjusted the data set until I got a t-stat above 2.” There are entire fields in economics, such as finding “anomalies” in asset prices, that seem fundamentally dishonest to me.

2. Romer seems to think that the intellectual dishonesty is mostly confined to a small group of Chicago School economists. [As an aside, although I went to the University of Chicago (as did Romer), I’m not the sort of Chicago School economist that he is referring to. My views on macro are closer to new Keynesian than real business cycle. I believe in sticky wages/prices.]

Romer might be right about the Chicago School. But can we trust his judgment on this issue? I’m not sure about that. In recent years I’ve found Keynesian claims in the blogosphere to be increasingly far-fetched. At times there seems to be willful denial of the obvious failure of the Keynesian model, both in terms of fiscal stimulus, and unemployment compensation. “Tests” are suggested, and when the results come in unfavorable to the Keynesian model, the test suddenly never happened. But I’m an anti-Keynesian, so I can’t really trust my own judgment on this issue. It’s much easier to notice flaws in those you disagree with, than those you agree with. Maybe that’s also true of Romer.

3. Here’s one piece of evidence that Romer may be somewhat lacking in the ability to judge intellectual honesty in others. In this post he provides a twitter exchange with a supporter of Lucas. At one point the supporter claims that Lucas was more honest than 99% of the profession. Romer interprets that claim as defending dishonesty; he suggests that the Lucas supporter is offering an “everyone does it” defense. But of course that’s not at all what the “better than 99%” claim means. To use a Christian analogy, it means something more like “we’re all sinners, but he sins less than almost anyone else.” Someone saying that is clearly not defending sin.

Elsewhere Romer portrays Stigler as a sort of Svengali who corrupted the Chicago school of economics. But in this post he clearly misunderstands what Stigler is saying. Stigler is making a positive claim (which seems accurate to me) not a normative claim.

And here Romer insists that there is such a thing as “objective truth”:

“My fundamental premise is that there is an objective notion of truth and that science can help us make progress toward truth.”

But a few paragraphs earlier sounds positively Rortian:

“In our discussions, claims that are recognized by a clear plurality of members of the community by as being better supported by logic and evidence are the ones that are provisionally accepted as being true.”

Yes, he can use “provisionally” as an out, but when are our beliefs no longer “provisional”? How big must the “plurality” be to make it “objective truth?” And is objective truth never wrong? Remember that we’re talking about macroeconomics here, how often do we have metaphysical certitude?

4. So far I’m just quibbling over minor issues of interpretation. And again, I think Romer is on the right side when fighting for intellectual honesty. But in this post I believe Romer goes far off course:

This description of the reputational equilibrium in science shows that it is inherently unstable in the face of entry by people with the norms of politics. If a small group of people with the norms of politics can establish themselves inside a science, they can destroy the equilibrium that sustains trust and communication. If the entrants guided by the norms of politics start publishing and citing Willie Horton papers, the scientists on the other side would be foolish to keep entering into a discussion with a presumption that “reasonable people can differ” and a willingness to be persuaded that some other position might be true. Politics begets politics.

The only way I can see to protect scientific discourse is to limit entry into the discussions of science. But this MUST NOT BE DONE on the basis of beliefs about what is true. It must instead be based on a demonstrated commitment to the norms of science. As part of this process of defending science, exclusion by shunning, plays an essential role. People who show, by publishing even one Willie Horton paper, that they are not committed to those norms, have to be excluded. So too must the people who promote and encourage Willie Horton papers. In science, “It was my PAC, not me” should not be an acceptable defense.

. . .

I think that the best course is to start by focusing on narrow specifics. In a paper:

1. Is a specific mathematical argument logically correct?
2. Do the words that the author uses to describe a mathematical result accurately convey what it shows?
3. Is the author giving a word or phrase a meaning that is diametrically opposed to the one that most readers will assume?
4. Does the author invoke the “Humpty-Dumpty” defense that a statement is true because “I use this word to mean something different from what everyone else thinks it means”?

These are simple questions of fact that we should be able to resolve. We can start on these first, but in this post, I am trying to be transparent about where I think that this process will should ultimately lead-to shunning the work of several prominent economists and their followers.

Here Romer is clearly talking about shunning people like Lucas, Prescott, and their followers.

I’ve meet many, many conservatives who tell me they’ve stopped reading Paul Krugman. I am often frustrated by Krugman posts, and occasionally feel he makes misleading arguments. But he also continues to do posts that are very insightful, and thus I feel an obligation to keep reading his blog. If Paul Romer stops reading brilliant economists because he disagrees with their methods in a single paper, then the problem of “epistemic closure” in the profession will only get worse. Shun the paper, not the person.

I’ve always tried to live up to the standards discussed by Romer, but I’m sure that I’ve fallen short here or there. On the other hand I also know for a fact that there have been times when commenters thought I was being dishonest, or purposely misleading, when that was not the case. [That’s the price one pays for being a contrarian.] And because of these false accusations, I know for a fact that if others followed Romer’s advice, they would have found at least one “Willie Horton” in my 1000s of blog posts. And Romer says that one is enough.

Actually, his strict standard can’t do much harm to people like Lucas and Prescott; they have their Nobel Prizes. It’s people like me that are much more at risk. I can see it now: “Sumner thinks the Great Recession was caused by tight money, not financial crisis. Only a crackpot or liar could make that claim.” I’m the sort of person that would be shunned, not Lucas. Maybe that’s why I feel so strongly about this issue.