The Perils of Amateur Epidemiology
By Arnold Kling
Robin Hanson cites one paper that says that higher unemployment reduces mortality, perhaps because of healthier eating, while another paper says that higher unemployment reduces the consumption of fruits and vegetables. He writes,
Either we can cross “eat healthier” off the list of possible ways unemployment helps health, or maybe fruits and veggies aren’t as healthy, and fast food as unhealthy, as we suppose.
Epidemiology is a difficult topic. I think that if economists are going to contribute anything to the field, it not be by doing studies based on regression methods. If anything, we should contribute skepticism about such methods, based on what we have learned about the biases caused by specification searches, data mining, and the bias toward publishing only “significant” effects (that bias itself undermines the reported significance). Instead, look for natural experiments, and don’t forget to be skeptical of those, too.
For example, this study of miscarriages was focused on the effect of caffeine intake. You can compare the incidence of miscarriages among caffeine drinkers and non-caffeine drinkers, but other factors may confound. The abstract says that it uses “multivariate analysis” to try to deal with confounding factors, such as smoking.
However, there is a possible natural experiment, based on nausea. Some women become nauseous during pregnancy, while others experienced much less discomfort. Women who become nauseous during pregnancy will greatly reduce their caffeine intake. If we see that nausea reduces the incidence of miscarriages for those women more than for women who don’t drink caffeine in the first place, then we can implicate caffeine as a factor in causing miscarriages.