How I Self-Police My Work
By Bryan Caplan
I have a long list of strange and extreme views, and I’ve been an arrogant hedgehog for as long as I can remember. As a rule, arrogant hedgehogs with lots of strange and extreme views are severely biased and grossly unreliable. Which raises two daunting questions.
The Reputational Challenge: Why should people take me seriously? Even if I happen to be correct, why would a reasonable person bother giving me a chance?
The Self-Referential Challenge: Why should I take myself seriously? Why should I consider myself so epistemically superior to the typical arrogant hedgehog with lots of strange and extreme views?
In all honesty, I take both challenges seriously. But it’s the self-referential challenge that weighs on me. I can endure the apathy of others, but not the idea that I’m living a lie. So what should I do? There are three basic paths:
1. Ignore the problem; just dogmatically assume I’m special.
2. Wallow in self-doubt; admit I’m an arrogant sinner and find another line of work.
3. Self-police; develop and follow epistemic procedures to offset my acknowledged shortcomings.
The latter option sounds best, of course – and it’s the latter option that I take. But the execution is crucial. How specifically do I self-police? Well, here’s how I self-policed the creation of my latest book:
Rule #1: Try to read very widely before writing. For starters, this means: Don’t search for stuff that agrees with you. Instead, search by topic, starting with Google Scholar. If you don’t understand a point, email the authors; most respond quickly. Above all, don’t limit your search to your own discipline. For The Case Against Education, for example, I went out of my way to read not just economics, but psychology, sociology, and education research. Without this interdisciplinary approach, I would have been painfully oblivious on dozens of crucial points.
Rule #2: Split big, interesting topics into small, boring topics. Sure, read “big think” pieces; but once you run out of those, subdivide the topic and apply Rule #1. When writing my latest book, I searched for research on scores of dull issues like “The effect of education on objective versus subjective health,” “Labor force participation and ability bias,” and “The effect of measurement error on estimates of the return to education.”
Rule #3: Mentally set your main thesis aside when you read about small, boring topics. Try to summarize each body of research on its own terms. You’ll never fully succeed at this, but it still helps keep you honest. In The Case Against Education, for example, my conclusions about IQ employment testing and political correctness sharply diverge from my ideological stereotype. I credit Rule #3 for shielding me.
Rule #4: Avoid controversial fundamental premises at all costs. If you can’t justify your position without making assumptions that would strike most people as implausible, go back to the drawing board. You can still have controversial conclusions, of course. But the point of an argument is to move from the known to the unknown. Thus, I tried to build my latest contrarian book on a well-trod foundation: the first-hand experience of being a student.
Rule #5: Energetically seek diverse feedback, especially from everyone you cite. Once I had a solid draft of The Case Against Education, I asked my RA to track down the email addresses for every single person in my References. About 75% were alive and accessible. Then I emailed every one of them the offer to read (a) the pages where I discuss their work, or (b) the entire book. About 15% took me up on at least one of these. As you’d expect, sympathetic readers were more likely to respond. But I also received and carefully read a big stack of criticism.
Rule #6: If someone says you mischaracterized their work, you almost certainly did. Experts are often wrong, but at minimum they know what they meant. Fortunately, even researchers who strongly disagreed with me gave me high marks for my reading comprehension. But whenever they had a nit to pick, I rewrote.
Do these rules enforce themselves? No. Do they guarantee accuracy if followed? No again. Yet they’re still great rules. For an arrogant hedgehog like myself, they are often bitter to follow. You know what? They should be bitter! The rules are designed to humble me before the truth, so I can do good work despite myself.
So this is how I try to cope with the self-referential challenge. And you know what? These rules for self-policing are also my best reply to the reputational challenge. Yes, I am an arrogant hedgehog with strange and extreme views. But unlike the vast majority of people in this questionable category, I’m not just self-aware of my flaws. I methodically struggle against them every day.