What Is IQ - and Why Does It Matter?
Years ago, I told Tyler Cowen, “It’s surprising that IQ tests predict life outcomes so well, because there’s usually no financial incentive to get a high score.” He replied, “People try out of pride – an under-rated motive.” So when Tyler blogged Duckworth et al, “Role of Test Motivation in Intelligence Testing” I naturally took notice. Key claims:
1. Material incentives boost IQ scores:
In 46 independent samples (n = 2,008), the mean effect of material incentives on IQ was medium to large: g = 0.64 [95% confidence interval (CI) = 0.39, 0.89], P < 0.001.
2. Material incentives have a bigger effect on the IQs of people with low scores:
Because exact baseline IQ scores were not reported in some samples, we created a binary variable where 1 = below average (i.e., IQ < 100) and 2 = above average (i.e., IQ ≥ 100). The effect of incentives was greater for individuals of below-average baseline IQ: Qbetween(1) = 9.76, P = 0.002. In 23 samples with IQ scores below the mean, the effect size was large: g = 0.94 (95% CI = 0.54, 1.35). In contrast, in 23 samples of above-average IQ, the effect was small: g = 0.26 (95% CI = 0.10, 0.41). A similar analysis in which baseline IQ scores (available for 43 of 46 samples) were treated as a continuous moderator indicated that a 1 SD increase in IQ is associated with about two-thirds of an SD decrease in the effect of incentives: b = −0.04, P < 0.001.
The authors reasonably infer that IQ is more of a composite intelligence/motivation measure than usually believed – especially by inter-disciplinary researchers. Their words to the wise:
Our conclusions may come as no surprise to psychologists who administer intelligence tests themselves (49). Where the problem lies, in our view, is in the interpretation of IQ scores by economists, sociologists, and research psychologists who have not witnessed variation in test motivation firsthand. These social scientists might erringly assume that a low IQ score invariably indicates low intelligence.
It’s hard to evaluate a piece like this without re-doing the underlying research, but the presentation is compelling and plausible. My main complaint is statements like this:
[W]e hypothesize that test motivation is a third-variable confound that tends to inflate, rather than erode, the predictive power of IQ scores for later-life outcomes.
This is especially odd given Duckworth et al’s effort to distinguish unobserved “true intelligence” from IQ. As far as I can tell, the authors do nothing to show that their results make IQ less predictive. They don’t even show that IQ is more mutable than earlier studies find; boosting incentives boosts scores while the incentives remain in place, but there’s no reason to think the boost lasts after the test-takers receive their pay. All the researchers require us to reconsider is the reason why IQ is so predictive and hard to durably improve.
For example, instead of saying, “IQ tests show that people are poor because they’re less intelligent – and intelligence is hard to durably raise” we should say, “IQ tests show that people are poor because they’re less intelligent and less motivated – and intelligence and motivation are hard to durable raise.” If, like me, you already believed in the Conscientiousness-poverty connection, that’s no surprise.
In any case, I urge you to read the original article. I’ve been reading IQ research and personality psychology for over a decade, but these results really are news to me. Your thoughts?