The Cutting-Room Floor
By Arnold Kling
Tyler Cowen rejoins the debate on the returns-to-schooling literature.
What’s striking about the work surveyed by Card is how many different methods are used and how consistent their results are. You can knock down any one of them (“are identical twins really identical?, etc.), but at the end of the day which are the pieces — using natural or field experiments — standing on the other side of the scale?
My guess is that “the cutting-room floor” is the answer to the question. The first time you run a regression, you usually get coefficients that you don’t like. You then proceed to “fix” the problem by correcting errors in the data, improving the specification, etc.
Next, suppose that you stick with the result that there is a small effect of schooling on earnings. Are you able to get it published, or does the journal editor leave it on the cutting-room floor?
Finally, suppose, like James Heckman, you are able to publish a result showing no large effect from schooling. Does Card include it in his survey, or does he leave it on the cutting-room floor?
Note that in Card’s survey, he points out that Angrist and Krueger appear to have backed off their use of the draft lottery as a natural experiment, because
In fact, the differences in education across groups of men with different lottery numbers are not statistically significant. Thus, the IV estimates are subject to the weak instruments critique of Bound et al. (1995), and are essentially uninformative about
the causal effect of education.
I would add that it would have been interesting to do the same study using women, who of course were not eligible for the draft. If you find that using birth date draft number for women produces the same effect as for men, you know that something is fishy.
My guess is that if you add up the cost of doing all of the studies in the education/earnings literature, it is far more than the cost would have been of undertaking a college scholarship lottery among a sample of students and observing the differences in outcomes. To study the relevant margin, choose the sample of students from those whose grades and SAT scores would barely gain them admission to second-tier state schools. Randomly give half of them full-ride scholarships and give half of them nothing. Observe their years of schooling and their subsequent earnings. Base your conclusion on those results.