How I Self-Police My Work
I have a long list of strange and extreme views, and I’ve been an arrogant hedgehog for as long as I can remember. As a rule, arrogant hedgehogs with lots of strange and extreme views are severely biased and grossly unreliable. Which raises two daunting questions.
The Reputational Challenge: Why should people take me seriously? Even if I happen to be correct, why would a reasonable person bother giving me a chance?
The Self-Referential Challenge: Why should I take myself seriously? Why should I consider myself so epistemically superior to the typical arrogant hedgehog with lots of strange and extreme views?
In all honesty, I take both challenges seriously. But it’s the self-referential challenge that weighs on me. I can endure the apathy of others, but not the idea that I’m living a lie. So what should I do? There are three basic paths:
1. Ignore the problem; just dogmatically assume I’m special.
2. Wallow in self-doubt; admit I’m an arrogant sinner and find another line of work.
3. Self-police; develop and follow epistemic procedures to offset my acknowledged shortcomings.
The latter option sounds best, of course – and it’s the latter option that I take. But the execution is crucial. How specifically do I self-police? Well, here’s how I self-policed the creation of my latest book:
Rule #1: Try to read very widely before writing. For starters, this means: Don’t search for stuff that agrees with you. Instead, search by topic, starting with Google Scholar. If you don’t understand a point, email the authors; most respond quickly. Above all, don’t limit your search to your own discipline. For The Case Against Education, for example, I went out of my way to read not just economics, but psychology, sociology, and education research. Without this interdisciplinary approach, I would have been painfully oblivious on dozens of crucial points.
Rule #2: Split big, interesting topics into small, boring topics. Sure, read “big think” pieces; but once you run out of those, subdivide the topic and apply Rule #1. When writing my latest book, I searched for research on scores of dull issues like “The effect of education on objective versus subjective health,” “Labor force participation and ability bias,” and “The effect of measurement error on estimates of the return to education.”
Rule #3: Mentally set your main thesis aside when you read about small, boring topics. Try to summarize each body of research on its own terms. You’ll never fully succeed at this, but it still helps keep you honest. In The Case Against Education, for example, my conclusions about IQ employment testing and political correctness sharply diverge from my ideological stereotype. I credit Rule #3 for shielding me.
Rule #4: Avoid controversial fundamental premises at all costs. If you can’t justify your position without making assumptions that would strike most people as implausible, go back to the drawing board. You can still have controversial conclusions, of course. But the point of an argument is to move from the known to the unknown. Thus, I tried to build my latest contrarian book on a well-trod foundation: the first-hand experience of being a student.
Rule #5: Energetically seek diverse feedback, especially from everyone you cite. Once I had a solid draft of The Case Against Education, I asked my RA to track down the email addresses for every single person in my References. About 75% were alive and accessible. Then I emailed every one of them the offer to read (a) the pages where I discuss their work, or (b) the entire book. About 15% took me up on at least one of these. As you’d expect, sympathetic readers were more likely to respond. But I also received and carefully read a big stack of criticism.
Rule #6: If someone says you mischaracterized their work, you almost certainly did. Experts are often wrong, but at minimum they know what they meant. Fortunately, even researchers who strongly disagreed with me gave me high marks for my reading comprehension. But whenever they had a nit to pick, I rewrote.
Rule #7: Be ready to bet on your beliefs. When you do bet, pay attention to the results so you can find out how accurate you are.
Do these rules enforce themselves? No. Do they guarantee accuracy if followed? No again. Yet they’re still great rules. For an arrogant hedgehog like myself, they are often bitter to follow. You know what? They should be bitter! The rules are designed to humble me before the truth, so I can do good work despite myself.
So this is how I try to cope with the self-referential challenge. And you know what? These rules for self-policing are also my best reply to the reputational challenge. Yes, I am an arrogant hedgehog with strange and extreme views. But unlike the vast majority of people in this questionable category, I’m not just self-aware of my flaws. I methodically struggle against them every day.
David R Henderson
Mar 26 2018 at 11:49am
Nicely done, Bryan.
My shorter version: The question I always ask myself–and I’m not saying I police as effectively as you–is “Am I right?” I always look at people’s criticisms even if they’re nasty and try to extract anything valuable from them.
Mar 26 2018 at 12:14pm
Rule 4 seems to be a theme of Michel Huemers work as well as your own. Is that why he’s your favorite philosopher?
Mar 26 2018 at 1:04pm
That is one very fierce looking hedgehog there.
Mar 26 2018 at 1:21pm
While you may have many strange and extreme views, you’ve never struck me as either arrogant or a hedgehog. Your posts (over the couple of years I have been following this blog) have always seemed polite and logically thought out. If people are saying you’re a prickly hedgehog, perhaps it’s a matter of projection.
Mar 26 2018 at 2:00pm
The fact that you are worried about this issue already puts you ahead of 95% of academics.
Mar 26 2018 at 4:14pm
Bryan, I always enjoy reading your thoughtful perspective, and I believe you prefer to be “right” more than you prefer to “think you are right”.
I believe Bryan is referring to being a hedgehog in the hedgehog vs fox context. He sees the world through the lens of a single defining idea.
Mar 26 2018 at 6:06pm
As one who has read ‘The Case Against Education’ and provided significant annotations of many sections you neglected to add Rule #8 (probably should be much higher up on this scale) – talk to people who are the subject of the book/article you are writing. This is also a key point that your GMU colleague and Washington Post columnist Steven Pearlstein offered in his review, “…Yet nowhere in the book is there evidence that he talked to real employers or real workers about how labor markets really work.”
Self-policing is fine but if it is done with blinders on it’s not worth much.
Mar 27 2018 at 12:15am
I think Alan makes a good point. I never believed school teaches anything much. It’s just crazy to think school teaches real skills. I’m surprised that people even believe it. I agree with pretty much all your arguments, but I think the explanation is darker. And that’s big deal. It’s important to talk to employers and employees. But I think it’s more important to have a real job for a while, than to talk to employers and employees. People aren’t going to tell you why they do what they do, because they are in denial. People are aligned by their cognitive biases, and so it’s hard for them to see the lies they tell themselves and others. So you got to watch them like how you watch animals in a zoo. Workplace brings out the worst in people. Having read your work for many years, I think it’s either sheer inexperience, or some “experience that you (and pretty much all human beings) lack” that stops you from understanding the labor market.
Mar 27 2018 at 5:05am
I have found, when commenting, the best approach is the basic syllogism. The two “problems” are that it requires a great deal of discipline to create a valid, clear, defensible syllogism and you are exposing yourself to tougher criticism because your thinking is easier to follow.
My undergraduate degree is in philosophy where discourse must be more rigorous to compensate for lack of empirical evidence. I never saw the same demands in economics even though the nature of the discussion was closer to philosophy than to empirical science.
Mar 28 2018 at 6:44am
Rule #4 reminded me of this Russell quote:
“the point of philosophy is to start with something so simple as not to seem worth stating, and to end with something so paradoxical that no one will believe it”
Mar 30 2018 at 11:57am
Concerning idea number 4. Dr Huemer has an idea about logic that if the premises seems right but the conclusion seems absurd, one might take a second look at the premise. Danny Frederick expanded this to include political systems. If the ideas seem right but the logical result seem atrocious then one might reconsider the original idea.
Sherlock Holmes mentioned something like this also in explaining his way of reasoning.
Comments are closed.