Warden: The part of the brain that remembers dance steps is also
the anger center. So, juveniles who know how to fox trot are 10% less
likely to commit a double homicide.
Prisoner: Who conducted this study?
Warden: The Institute of Shut Your Fat Face!
–The Simpsons, “The Wandering Juvie”
Whenever non-academics tell me that “studies show X,” I cringe. It’s not because I’m skeptical of empirical research. It’s because the “chain of custody” from researchers to the man in the street is multiply broken. Scholars publish results, some popularizers mis-summarize them, other popularizers pick and choose the mis-summaries they want to trumpet, and the public remembers the trumpeted mis-summaries that comfort and/or terrify them.
And that’s when the system is “working”! I’ve also noticed that non-academics just let their imaginations run wild, then try to make their say-so stick by alluding to “studies.”
If this depresses you, let me point out that matters are much better than they used to be. It is now ridiculously easy to (a) call shenanigans on vague references to “studies” and (b) find out what studies actually show. Using Google Scholar, for example, you can quickly locate the main serious research on any topic of interest.
To pick a topic at random, I decided to look into the health risks of oral contraceptives (OC). I’ve often heard vague talk about blood clots and breast cancer being “linked” to OC, followed by heavy-handed insinuations that OCs are too dangerous to use. If that is really the issue, though, the obvious question to ask is simply, “Do OCs raise overall mortality?” A quick Google Scholar search of “oral contraceptives mortality” reveals multiple large high-quality studies, all of which find that OCs do not raise mortality. Searches for the link between specific ailments and OCs produce less clear-cut results, but also reveal plenty of studies claiming health benefits of OCs.
Most complaints about “studies” are rooted in agnosticism. Who can really know what’s true and what’s not? My point is that agnosticism is misplaced. Yes, blanket doubt is better than falling for fashionable nonsense. But not only is the Truth is Out There; it’s a lot closer than you think.