The Hill Criteria and the Stimulus
By Arnold Kling
This week’s IGM forum is on whether the stimulus produced more jobs. Most of the panelists agree that it did. I wonder how many of the panelists are familiar with the Bradford Hill Criteria for making inferences based on observational data.
Hill wanted to convince people that smoking caused lung cancer. The available data was not experimental. Instead, it was observational data, where there is a risk of interpreting a coincidence as causation. So, he set forth some criteria to justify the causal inference.
A good question would be whether these criteria are satisfied in macroeconomics, where we have observational data to use in trying to determine whether a fiscal stimulus has positive effects. In fact, the criteria are not satisfied very well at all.
The first criterion is “strength of association.” This one fails because the usual association between fiscal deficits and the economy is that the economy is weak when deficits are high. This can be attributed to the fact that when the economy is weak, tax revenues decline and spending on safety-net programs rises. Still, it wrecks “strength of assoication.”
Another criterion is consistency. Do we nearly always see fiscal stimulus working and fiscal austerity causing higher unemployment? No.
Another criterion is “dose-response” relationship. Do we see that larger deficits almost always create larger increases in employment? No. (We see a dose-response relationship in macroeconomic models, but that is not the same thing as observing it in the real world.)
Two other criteria are plausibility and coherence, which I see as assessing how the causal narrative relates to other widely-accepted theories. If it is reinforced by other theories, it is plausible. If it is contradicted by other theories, it raises problems of coherence. For people who are Keynesian to being with, the relationship between deficits and employment is plausible and coherent. Those of us who find non-Keynesian economics more persuasive would take the opposite position.
For more on the methodological problems associated with empirical macroeconomics, see this paper.