Value-Added and Social Desirability Bias
By Bryan Caplan
Chetty, Friedman, and Rockoff’s research on teacher value-added is widely interpreted as an argument for giving good teachers a fat raise. Obama’s 2012 State of the Union speech pulls a dollar figure right out of their abstract:
At a time when other countries are doubling down on education, tight
budgets have forced states to lay off thousands of teachers. We know a
good teacher can increase the lifetime income of a classroom by over
$250,000. A great teacher can offer an escape from poverty to the child
who dreams beyond his circumstance. Every person in this chamber can
point to a teacher who changed the trajectory of their lives. Most
teachers work tirelessly, with modest pay, sometimes digging into their
own pocket for school supplies — just to make a difference.
Teachers matter. So instead of bashing them, or defending the status
quo, let’s offer schools a deal. Give them the resources to keep good
teachers on the job, and reward the best ones.
When I finally read the original paper, I was bemused to discover that Chetty et al.’s support for merit raises is muted at best.
Using Monte Carlo simulations analogous to those above, we estimate that retaining a teacher at the ninety-fifth percentile of the estimated VA [value-added] distribution (using three years of data) for an extra year would yield present value… [of] $212,000. In our data, roughly 9 percent of teachers in their third year do not return to the school district for a fourth year. The attrition rate is unrelated to teacher VA, consistent with the findings of Boyd et al. (2008). Clotfelter et al. (2008) estimate that a $1,800 bonus payment in North Carolina reduces attrition rates by 17 percent. Based on these estimates, a one-time bonus payment of $1,800 to high-VA teachers who return for a fourth year would increase retention rates in the next year by 1.5 percentage points and generate an average benefit of $3,180. The expected benefit of offering a bonus to even an excellent (ninety-fifth percentile) teacher is only modestly larger than the cost because one must pay bonuses to (100 − 9)/1.5 ≈ 60 additional teachers for every extra teacher retained.
The policy that dramatically passes the cost-benefit test is “deselection,” better known as firing bad teachers.
To quantify the value of improving teacher quality, we evaluate Hanushek’s (2009, 2011) proposal to replace teachers whose VA ratings are in the bottom 5 percent of the distribution with teachers of average quality.
Rothstein estimates that a policy which fires teachers if their estimated VA after three years falls below the fifth percentile would require a mean salary increase of 1.4 percent to equilibrate the teacher labor market.43 In our sample, mean teacher salaries were approximately $50,000, implying that annual salaries would have to be raised by approximately $700 for all teachers to compensate them for the additional risk. Based on our calculations above, the deselection policy would generate NPV gains of $184,000 per teacher deselected, or $9,250 for all teachers on average (because only 1 out of 20 teachers would actually be deselected). Hence, the estimated gains from this policy are more than ten times larger than the costs. [emphasis mine]
What’s up? I once again point my accusatory finger at Social Desirability Bias. Rewarding good teachers sounds a lot nicer than firing bad teachers. So when research comes along that potentially recommends both, pundits and politicians don’t coolly crunch the numbers. They leap to the recommendation that’s pleasing to the ear. So what if the original researchers find that firing bad teachers wins with flying colors? Move along folks, nothing to see here…