Why do natural experiments matter? One of the toughest problems in economic research is figuring out whether a relationship between two variables is causal or coincidental. So, for example, economists find that the lifetime earnings of people who go to school for 12 years are higher than those of people who go to school for 11 years. But what if those who stay in school longer are more motivated or smarter than those who are in for only 11 years? Then the earnings of the more-schooled would be higher even if schooling per se doesn’t add much to earnings. What one would ideally like is to compare the earnings of people whose motivations and intelligence don’t differ.
Enter compulsory schooling. In 1991, Mr. Angrist and the late Alan Krueger noted that under compulsory-schooling laws, students born in the first quarter of the calendar year would be able to leave school earlier than students born in the fourth quarter. Sure enough, they found, those born in the fourth quarter had an average of 0.15 year more in school. And the earnings of those in the fourth quarter were 1.4% higher than the earnings of those born in the first quarter. Extrapolate that to a full-year difference in schooling, and you can conclude that one extra year of schooling raises earnings by about 9%.
This is from David R. Henderson, “‘Natural Experiments’ Lead to an Economics Nobel,” Wall Street Journal, October 11, 2021. (The print edition will be tomorrow, October 12.)
I would quote more, especially about the controversial Card/Krueger finding on the minimum wage, but my contract with the Journal allows me to quote only 2 paragraphs until 30 days is up. I’ll post the whole thing on November 11.
Thanks to Alex Tabarrok for taking a look at the draft, although the section he wisely suggested I add, on Card’s study on the Mariel Boatlift, was cut.
I’ll publish a couple of the deleted sections tomorrow.
READER COMMENTS
zeke5123
Oct 11 2021 at 8:43pm
I wonder if this study really is a “win” for the human capital model of education (as it appears) or if it really is just telling us about signaling. Assume that performance and age are positively correlated (at least under 30).
Therefore, all other things equal, seniors who are older will do better compared to seniors who are younger.
Seniors who do better in high school get into better colleges. By getting into better colleges, they might have slightly better opportunities. Add that up over a lifetime, and I could see a relatively small bump.
I guess this is the hard problem with economics. Even when you have what appears to be a natural experiment, there are different explanations because it is really hard to hold all else equal.
David Henderson
Oct 11 2021 at 9:17pm
If I understand Bryan Caplan’s work on signaling, and I think I do, these results are consistent with his model. The issue isn’t whether additional schooling leads to higher incomes but why additional schooling leads to higher incomes.
robc
Oct 12 2021 at 9:13am
I think to break it apart, what we need is those born in Q1 who still dropped out vs those born in Q4 who still dropped out. It wasn’t clear from your excerpt if the comparison was that or all Q1 vs Q4.
My understanding of Caplan’s theory is that it is the granting of the degree, the finishing, that is the signal and responsible for most of the gains.
This natural experiment should give us the breakdown to how much of the increase in earning is due to time in school and how much is due to degree. And also of interest, is there a difference between earning of Q1 born who finish and Q4 born who finish?
Does being slightly older help? It does in sports, as it has been found that in sports with a consistent age cutoff date (like Jan 1), those born in Jan/Feb/Mar are significantly more likely to become pros than those born in Oct/Nov/Dec. For baseball, the cutoff is usually Aug 1, so should mean an advantage to Aug/Sept/Oct babies, but I haven’t looked into that. The Jan 1 was for soccer in Europe and apparently there is a real difference.
Mark Z
Oct 11 2021 at 11:01pm
I imagine one could test this by looking at whether the time of year of one’s birth is still associated with income even when just looking at students who complete high school, right? In fact, just controlling for academic performance while looking at the effect when one is born would, I think, be enough to rule out this specific hypothesis. I’m guessing Angrist and Krueger did the latter, as it seems like a rather obvious confounder.
zeke5123
Oct 12 2021 at 12:43pm
That would seem the way to tease it out, yes. I am curious if they did. Will try to track it down and report back.
Knut P. Heen
Oct 12 2021 at 6:03am
Natural experiments are not controlled experiments and cannot be used to infer causality.
A few years back, Norway Chess (a classical chess tournament in Norway) made a change to the rules. Every drawn match would be decided by a game of Armageddon. The result was that almost every game ended in a quick draw, and were decided by Armageddon. The experiment seems to indicate that the introduction of Armageddon leads to more quick draws, a big surprise! However, the reason for the quick draws was an influenza outbreak at the tournament. The players were sick and preferred to go back to their hotel rooms to rest.
Natural experiments suffer from the same problem. Some unobserved variable may be the real cause of the change. If empiricists looked at the data from Norway Chess, they would see a change in the rules and more quick draws. They would not see the health status of the players (because that data is not recorded).
Lawrence
Oct 12 2021 at 8:18am
David, in the last paragraph of the WSJ article, you cited the findings about Medicaid recipients. Did you have any thoughts on why there were no significant outcome improvements between those who received Medicaid and those who didn’t?
Mark Young
Oct 12 2021 at 9:46am
Are there any jurisdictions where the cut-off date for starting school is different from its neighbours? That should provide a natural experiment.
Also changes in the cut-off date might be interesting. Nova Scotia changed from October 1st to January 1st in 2008, so the first class under the new rules graduated last year. https://novascotia.ca/news/release/?id=20080111001
Quebec and Saskatchewan still have different start dates from other provinces.
Dylan
Oct 12 2021 at 4:10pm
Not really a neighboring jurisdiction, but I moved to Hawaii in elementary school and I know the cutoff there was January vs. September in the state I started school in. I was born in December, so I became one of the older students in my grade. For sports, the cutoff of age remained, so I was on teams with people that were all in older grades than me. Being smallish for my age already, that didn’t help things, particularly because they grouped two grades worth of people together, so in 4th grade all the other people on my soccer team were in 5th and 6th grade.
Manfred
Oct 12 2021 at 10:09am
Sorry, but I am not celebrating this Nobel. Hate to be a Debbie Downer, but somebody will need to explain to me why this is not a Nobel to the Free Lunch, a concept which I thought we economists thoroughly reject. One can their analyses in fancy equations, fancy wording, complicated terms, august publications in the very woke AER and Econometrica, but at the end, it is a celebration of the Free Lunch. Gone are the days when economists actually pointed out trade-offs and incentives and won Nobel Prizes for it. And gone are the days of Arnold Harberger’s speech “The Economist and the Real World”.
Knut P. Heen
Oct 13 2021 at 5:58am
The price is for the method, not the conclusions. I know some people in the committee, and they are certainly not woke.
Comments are closed.