Here’s an excerpt from chapter 2 of the current draft of The Case Against Education.
Making You Smarter
While educators often promise to teach students how to think, they rarely vow to raise students’ intelligence. Trying to “make your pupils smarter” smacks of hubris. However, when you look at the data on IQ – psychologists’ standard measure of intelligence – education matters. Summer vacation, intermittent attendance,delayed school entry, and dropping out all measurably depress IQ.[1] Some experimental early childhood programs have increased IQ by over 30 points – moving kids’ performance from roughly the 2nd percentile to the 50th percentile of their age group.[2] Studies that carefully measure students’ time show that IQ rises more on school days than non-school days.[3] Isn’t this conclusive evidence that education makes us smarter?
Not really. While the facts are secure, the interpretation is shaky. The first major worry: People can sharply improve on virtually any test by practicing – and a little practice goes along way. A major review article pulled together fifty relevant studies of practice on cognitive tests. On average, “a candidate who scored at the 50th percentile on the first test could be expected to score at the 60th percentile on the second test and at the 71st percentile on the third test.”[4] Explicit coaching – “teaching to the test” – works even better.[5] When students repeatedly take identical versions of the same test, their scores skyrocket.[6]
A cock-eyed optimist might rejoice that mankind is only a few hours worth of practice away from massive intelligence gains. This optimism, however, leads to absurdity: Can you transform average students into geniuses by handing them the answer key before their IQ test? Most researchers draw the sobering conclusion that test preparation yields only “hollow gains.”[7] Preparation inflates measured intelligence without raising genuine intelligence.[8]
The fact that test preparation yields large but hollow gains hardly shows that all large gains are hollow. Still, the power of preparation should make us suspicious. Maybe education raises IQ because education is a dilute form of IQ test preparation. As psychologist Steven Ceci explains:
It is through direct forms of instruction… that children learn the answers to many of the questions that appear on a popular IQ (and other aptitude) tests. For example, within a given grade level there is a correlation between the total number of hours of schooling a child receives and scores on verbal and mathematical aptitude tests. Similarly, there are negative correlations between the total number of teacher or student absences and scores on such tests. Also, quantitative and language-related scores are strongly correlated with the length of the school day and with the actual amount of time on task, beginning in first grade. So it makes intuitive sense that much of the knowledge that aptitude tests, including IQ, tap is accumulated through direct encounters with the educational system. Answers to questions on the WISC-R, such as “In what continent is Egypt?”; “Who wrote Hamlet?”; “What is the boiling point of water?”; and “How many miles is New York from L.A.?” are probably learned through direct teaching methods. Teachers may not be aware that they are teaching answers to questions on IQ tests, but this is precisely what they are doing in their history, reading, literature, geography, and math classes…[9]
Ceci also notes that schools teach students to offer the kinds of answers IQ tests favor. How are an apple and an orange alike? IQ tests award only partial credit for such factually correct answers as, “They’re both round,” “They’re both edible,” or “They both have seeds.” For full credit, you have to say, “They’re both fruits.” School also makes students sit still and pay attention.[10] These habits aren’t just crucial for test-taking; they’re useful life skills. Yet sitting still and paying attention aren’t “intelligence” in any normal sense of the word.
If education truly raised intelligence, education would enhance performance on all sorts of cognitive challenges – in and out of the classroom. In reality, though, IQ gains yield spotty payoffs even on narrowly academic subjects. Probably the best study of the effect of education on IQ looks at the scores of over one million 18-year-old Swedish men.[11] The researchers know each student’s exact age and test date, so they possess an unusually precise measure of how many days they spent in and out of school. Their major finding: school days noticeably raise scores on synonym and technical comprehension subtests without raising scores on spatial and logic subtests. The authors infer that education raises “crystallized intelligence” but not “fluid intelligence.” A better interpretation, though, is that education improves some specific skills without increasing intelligence at all. Considering how little students usually learn, the measured effect of Swedish education on the synonym and technical comprehension subtests is impressive. Still, to equate subject-specific gains with higher intelligence smacks of double-counting.
Worries about “hollow IQ gains” are admittedly a tad philosophical. The other major worry about the effect of education on IQ, however, is completely pragmatic. Suppose for the sake of argument that IQ were a perfect measure of genuine intelligence. When IQ goes up, genuine intelligence automatically rises in sync. Even in this scenario, a large effect of education on IQ would only be impressive if it were durable. In the short story “Flowers for Algernon,” a mentally retarded man named Charlie Gordon receives an experimental treatment to cure his disability.[12] Charlie’s intelligence eventually rises to the level of genius. Tragically, though, the transformation is short-lived. By the end of the story, all of Charlie’s intellectual progress evaporates. In one sense, the experiment worked. In a deeper sense, the experiment failed.
“Flowers for Algernon” is science fiction, but life mirrors art. Making IQ higher is easy. Keeping IQ higher is hard. Researchers call this “fade-out.” We see fade-out in early childhood education programs. After six years in the famous Milwaukee Project, experimental subjects’ IQs were 32 points higher than controls’. By age fourteen, this advantage had declined to 10 points.[13] In the Perry Preschool program, experimental subjects gained 13 points of IQ, but all of this vanished by age 8.[14] Head Start raises pre-schoolers’ IQs by a few points, but the gains disappear by the end of kindergarten.[15]
You could object that pre-schoolers are unusually likely to forget what they learn. The pattern, however, extends all the way through high school. Extensive research on “summer learning loss” compares students’ scores at the end of one school year to their scores at the beginning of the next school year. The average student intellectually regresses roughly one full month during a three-month summer vacation.[16] The older the students, the steeper their decline. For reading, to take the clearest case, first- and second-graders actually slightly improve over the summer. By the time students are in middle school, however, one summer vacation wipes out over three months of reading proficiency.[17]
Educators tend to see summer learning loss as an argument for year-round school. If summer makes students stupid, let’s abolish summer. The key flaw in this argument: You can’t keep kids in school forever. Everyone graduates eventually. Once you graduate, you’re no longer in school – and learning loss kicks in. To quote “Tiger Mother” Amy Chua, “Every day you don’t practice is a day that you’re getting worse.”[18]
Does education have any effect on genuine intelligence? Despite decades of research, we really don’t know. What we do know is that education has far less effect than meets the eye. The effect of education on intelligence may not be entirely hollow, but it is largely hollow. The effect of education on intelligence may not be entirely temporary, but it is largely temporary. School might permanently make you slightly smarter. As we shall see next chapter, though, this can’t explain more than a sliver of the effect of education on income.
[1] See especially Ceci, “How Much Does Schooling Influence General Intelligence and Its Cognitive Components?,” pp.705-8.
[2] Arthur Jensen, The g Factor, pp.333-44.
[3] Carlsson, Dahl, and Rooth, “The Effect of Schooling on Cognitive Skills”; Steltz et al, “The Effect of Schooling on the Development of Fluid and Crystallized Intelligence”
[4] Hausknecht et al, “Retesting in Selection,” p.381.
[5] Hausknecht et al, “Retesting in Selection,”p.380-1. Commercial claims about the effectiveness of SAT preparation classes are however grossly overstated: see e.g. Powers and Rock, “Effects of Coaching on SAT I.”
[6] more cites
[7] See e.g. Arthur Jensen, The g Factor, pp.333-44; Nijenjuis, Voskuiji, and Schijve, “Practice and Coaching on IQ Tests: Quite a Lot of g” + others
[8] Freund and Holling, “How to Get Really Smart”
[9] Ceci, “How Much Does Schooling Influence General Intelligence and Its Cognitive Components?,” p.717 (referenced omitted).
[10] Ceci, p.718.
[11] Carlsson, Dahl, and Rooth, “The Effect of Schooling on Cognitive Skills”
[12] cite
[13] Jensen p.341; check w/Garber
[14] Barnett, “The Effectiveness of Early Childhood Intervention,” p.975.
[15] Barnett, p.976.
[16] Cooper et al, “The Effects of Summer Vacation on Achievement Test Scores.” Most of this research focuses on “achievement tests” rather than “IQ tests,” but the content closely overlaps. Math and reading questions are staples on both kinds of tests. [Also get Heyns stuff on IQ specifically?]
[17] Cooper et al, p.259.
[18] Battle Hymn of the Tiger Mother, p.86. Chua is approvingly quoting her daughter’s violin teacher.
READER COMMENTS
Brian Moore
Jan 29 2013 at 10:55am
Wait, IQ tests have factual information on them? How does knowing who wrote Hamlet measure your IQ?
I don’t remember ever taking an IQ test, but I always figured they had logical/problem solving questions, not just memorization stuff.
TK
Jan 29 2013 at 11:03am
Every time I see the word “education” in this post, I wonder whether you really mean “school”. All of the criticisms seem to focus on school. Does your “case against education” also encompass self-teaching, apprenticeship, discipleship, parental teaching, personal growth, and so on?
Tracy W
Jan 29 2013 at 11:14am
Aren’t aptitude tests meant to measure someone’s potential to do something, as opposed to achievement tests, which measure whether someone can do something?
If that’s the case, then the failure of education to persistently raise people’s performance on aptitude tests is an argument (not a conclusive argument, just an argument) that the psychologists who are designing aptitude tests are doing something right.
Education should be about raising students’ performance on achievement tests.
Dave
Jan 29 2013 at 11:29am
If IQ test scores can be affected by preparation, I’m suspicious that the tests are actually measuring what they purport to measure (or perhaps more fairly, what they are commonly thought to measure by the general public).
From this excerpt, it seems to me that not only is there not compelling evidence for the effect of education on genuine intelligence, we don’t even have a solid grasp on what, if anything, the term “genuine intelligence” really means.
gwern
Jan 29 2013 at 12:07pm
> Wait, IQ tests have factual information on them? How does knowing who wrote Hamlet measure your IQ?
People who are smarter are more likely to know who wrote Hamlet. Similar to vocabulary being a useful measure of IQ: even if people are exposed to new obscure words the exact same number of times, the smarter people are more likely to remember the word after just a few exposures and know what it means.
All perfectly useful correlates with IQ – as long as you don’t conclude ‘I can make kids smarter by making them memorize vocab!’ and create hollow gains (destroying the correlation).
Mr. Econotarian
Jan 29 2013 at 12:37pm
I think it is recognized that IQ tests can have a motivational bias.
“students first took an IQ test under normal conditions, and then took it a second time, rewarded immediately with an M&M for each question they got right. These students demonstrated a 1.8SD improvement in mean IQ, indicating that motivation can play a huge part in intelligence and achievement. However, the study also reveals that students who did not respond to the M&M incentive went on to achieve better outcomes, including higher GPAs, graduation rates, and degree attainment. It is these students who demonstrate true grit, and perform well simply because that’s what they want to do regardless of the incentive.”
http://www.onlinecolleges.net/2012/11/07/10-things-every-student-needs-to-know-about-grit/
I myself was a “low-grit” student, in that I was a far more interested in researching what I wanted to research than that offered to my by my school. I am glad that I managed to stick with it enough to pick up the formal subjects, but was a “B” student.
English Professor
Jan 29 2013 at 1:16pm
“Summer vacation, intermittent attendance, delayed school entry, and dropping out all measurably depress IQ.”
Is this statement accurate? Those things certainly affect acquired learning–that is, as you explain in the post, students clearly forget a lot over summer vacation, and if they drop out or have less schooling overall, they will have less chance to acquire organized information. One would expect these students to test more poorly on things like the SAT than students who have had more formal education. But are there really measurable differences in student IQ if the test is taken in May (before vacation) rather than in September (after vacation)? This strikes me as highly unlikely.
Brian
Jan 29 2013 at 2:22pm
Interesting commentary.
On one level, one wouldn’t expect IQ to change at all because of schooling, IF IQ is really measuring intelligence. As Tracy W points out, education really aims at improving achievement.
On the other hand, the Flynn effect is well known and probably real, with one possible explanation for continued increases in IQ being that people are now better at thinking abstractly than they use to be. One could surely imagine education and school affecting that.
In my own B.S. program, we see nominal increases in IQ of our students from beginning to end as measured by SAT and GRE tests. This effect is not likely due to test prep, nor teaching to the test, nor due to in-school/out-of-school effects. And unlike Bryan’s statement at the top, we have enough hubris to claim that we make our students smarter.
What does it mean? Who knows? Ultimately, relative to other students applying for grad shool, our students improve more in their ability to answer math questions than they lose in verbal questions. This is good since we consider math to be more important.
As a side note, the GRE/SAT effect is a great argument against Bryan’s signaling model of education. In general, students who study technical/mathematical subjects improve their percentiles on the GRE math section (over the SAT) but decline in the verbal section. This is reversed for students in the humanities. It’s clear that choice of major has a significant effect on general intellectual skills (not on just the obvious factual knowledge), undercutting the notion that college is all about signaling.
Stuart Buck
Jan 29 2013 at 3:08pm
Interesting stuff. Still, one reaction may be who cares? Few things have a permanent effect (unless it’s disabling). There’s no vitamin or drug you can take that lasts the rest of your life after taking it once. And I can personally attest that you can train to run a marathon, but if you stop doing the long runs, it will be a relatively short time before you lose all or nearly all of your endurance. But none of that says anything about the benefit of running or taking vitamins if you do so consistently.
Ken B
Jan 29 2013 at 3:48pm
@Brian Moore: IQ tests are all about correlations. If question X is highly correlated to successfully completing college for example, it could be used as part of a college admission test. (Even if the questions is “what is the response to ‘Marco’?” or something equally odd.) So, yes factual questions can make perfectly good sense on IQ tests if they have strong correlation to whatever proxy for IQ you are using.
Interestingly enough this objection was raised about questions in the past alleged to have a racial bias. In most cases though the questions, when you looked into the results, *favored* the minority it was alleged to disfavor. The reason is, we judge questions by whether they *seem* sensible or racial in tone, but the test constructors use correlation coefficients.
IQ testing is really a very complex area and most of what you read is out of date and misleading.
(Including doubtless me. I understand the general principles. But I looked into this pretty well 20 years ago, and not since. So my detailed knowledge is 0 at this point.)
Joseph Hertzlinger
Jan 29 2013 at 4:19pm
Preparation can affect physical fitness tests. There’s even fade-out. Does this means they don’t measure physical fitness?
Philo
Jan 29 2013 at 4:19pm
Re: “The effect of education on intelligence may not be entirely hollow, but it is largely hollow. The effect of education on intelligence may not be entirely temporary, but it is largely temporary.”
If ‘hollow’ is a synonym of ‘temporary’, then these two sentences are a redundancy.
If ‘hollow’ means the same as ‘illusory’ or ‘non-existent’, then there is a tension between two sentences. A better phrasing would be: “The apparent effect of education on intelligence may not be entirely illusory, but it is largely illusory. And what little effect there is may not be entirely temporary, but it is largely temporary.”
MingoV
Jan 29 2013 at 5:43pm
Raising IQ: The evidence I’ve seen is that heredity determines maximum intelligence, but attained intelligence can most easily be increased during the first few years of life. From birth to age 5 or so, the brain grows, is very plastic (can rearrange structure and neural connections), and greatly multiplies the average number of connections per neuron.
The best way to improve intelligence is to keep infants and toddlers healthy, provide good nutrition, and give them a variety of learning experiences: listening, viewing, touching, smelling, and exploring. Initially, the learning experiences will be passive, but progression to independent learning provides more benefit.
Steve Sailer
Jan 29 2013 at 7:06pm
“Even in this scenario, a large effect of education on IQ would only be impressive if it were durable.”
Actually, I think temporary increases could get children over a variety of crucial hurdles, such as learning to read or learning various math skills. If you can get somebody’s IQ up from 70 to 80 long enough for them to learn how to read, they will have a better life even if they go back to being a 70 IQ person who never reads or writes anything other than Youtube comments. It’s better than being an illiterate.
Steve Sailer
Jan 29 2013 at 7:09pm
I’ve been reading for decades about how studies show you can’t really beat the test through massive test prepping.
But the Tiger Mother theory is that you can, and that the academic studies are trivial compared to what they put their own kids through. Judging by the surnames of National Merit Scholars these days, I’m not so sure I believe the experts over the Tiger Mothers.
Clay
Jan 29 2013 at 10:18pm
The core points are something you expect to hear from an obnoxious student at a typical high school, but here they are developed into shockingly well researched and convincing arguments. If Bryan can sustain this type of entertaining and insightful argument for the length of the book, this will make quite a splash.
Tracy W
Jan 30 2013 at 6:26am
@Ken B: Aren’t you talking about achievement tests rather than aptitude tests? To take your example, how many people complete college without having first learnt to read and write (be that by attending a formal school, or home schooling, or whatever?) Or complete college if they’re not familiar with the language it’s being taught in?
Personally I think that someone who managed to complete a course at an American college after starting while not knowing who Shakespeare is, and with a very limited English vocab is more likely to be someone who has a higher IQ than someone who completed college after performing highly on such test measures.
Keith K.
Jan 30 2013 at 8:10am
So when can we expect this bad boy to hit the presses?
Silas Barta
Jan 30 2013 at 3:29pm
True story: when I was in second grade, and the teacher was preparing us for IQ-like tests, she showed one example problem, which was:
“Which of these is not like the others?
A) Baseball player with cap
B) Cowboy with hat
C) Policeman with hat
D) Clerk without hat”
You’re supposed to say D), but before I knew that, my answer was “B), because that’s the only one that’s, like, not a real job anymore”.
How is that an invalid/unintelligent answer [assuming what I had been told about the status of the cowboy is correct]?
Also, seriously, remove the ridiculous historical/word knowledge questions from IQ tests. What’s the deal with that?
[edited with commenter’s permission–Econlib Ed.]
Floccina
Jan 30 2013 at 4:20pm
To answer commentator’s objections above:
I think that Bryan’s work shows that the schools should focus on very practical and useful skills that help the students live better lives BUT THEY DO NOT.
Skills like how to show up for work on time, how to recognition some of the bigger frauds that are around, how to repair a car, fix plumbing, some simple principles of chemistry and physics, a little about the probability of winning the lottery but these are squeezed out by signaling meant to prepare the student not for life but for further schooling. How often do think the average person needs to factor a polynomial?
Lest you think this is BS, Obama has said he wants everyone prepared for college and he is not alone.
Comments are closed.