Lots of great feedback in response to last week’s question, “Why don’t applicants volunteer their test scores?” I’m increasingly impressed by the wide range of first-hand education/job experiences; clearly the world is full of puzzles few economists have ever conceived, much less addressed.
Several respondents doubted the usefulness of applicants’ test scores. But this contradicts a large literature finding that IQ-type tests are among the very best predictors of job performance for a very wide range of occupations.
A few highlights from the comments:
1. When I graduated in 1998, a lot of high-test hiring companies like finance and consulting did ask for SAT scores.
[…]
4. HR departments hammer home that you’re not supposed to make decisions based on information you’re not supposed to know.
5. I have seen something similar happen, where applicants who were
disabled would volunteer information about their condition that
suggested they would not need any more sick time or “accommodations”
than a non-disabled person… Anecdotally it tended to
benefit the discloser.
I recently served as the hiring manager on a government job for which
most of the applicants were recent graduates. I received a few hundred
applications, of which 5-10 percent provided SAT or GRE scores. Those
scores typically put the applicant in the top couple percent of test
takers and were frequently from Ivy League schools.In spite of recognizing their accomplishment, I took the scores as a
negative signal in the vein of Prof. Barnett. Did they have nothing
relevant to the job for which they were applying that they could be
using to fill that space?
As an entry-level actuary out of college I provided my (what I
thought were impressive) test scores, and I was hired (not as a result,
but not not as a result?)I didn’t think it was socially awkward at the time, but that was more
my naivete than an astute observation. I subsequently took the scores
off of my resume.
Bob Knaus really made me think:
It worked for me, in 1996. As a high-school dropout with good tech
experience but no credentials, SAT scores of 750 verbal and 735 math got
me a position as a management consultant. Saved me all the time and
expense of a college education.
The decisions to show/not show scores is based entirely on the social
context for this. In industries like strategy/management consulting, it
is not uncommon (even for many partners) to show their GMAT/LSAT/SAT
scores. When you’re selling generalist “smarts,” you’re expected to use
anything to your advantage.When you’re selling technical skills, you show OTHER qualifications –
certifications (look at how popular PMP certs are in consulting… I
have yet to find someone who was transformed into a better manager
because of it…).
I conducted a natural experiment a few years back on this. I was looking
for a job, competing against ivy league types. Initially, my call back
rate was about 5 percent. Once I put my GRE score on my resume, my call
back rate jumped to 80 percent (for targeted applications). I wouldn’t
argue that this works for all jobs, and now my experience trumps test
scores, but in the investment industry, it makes a difference.
First, I don’t believe you are allowed to rely on such information
whether it is volunteered or asked for. I once mentioned my immigration
status and another time I told my interviewer that I was married. (both
are protected categories) It was in passing but the interviewer’s
reaction was the same. They were obviously shaken and made a point to
tell me that it was not relevant to the process. Of course, a small
organization could make those decisions informally and take such
information into consideration. But an organization of any significant
size will need explicit hiring policies.
The question employers face isn’t “what the law says” but “what you can get away with.” I can believe it’s harder for large organizations to unofficially break the law. But even large organizations ultimately have human being(s) – not “hiring policies” make the final decision. On “obviously shaken” interviewers, I’ve heard virtually the opposite: Even lawyers applying for positions in labor law say that interviewers habitually ask illegal questions. Few humans outside of North Korea manage to scrupulously avoid forbidden small talk.
GMU’s own Dan Klein:
Suppose Sally is submitting a resume to employers. Suppose she has
really high SAT scores, and that she has skipped going to college.Could Sally overcome the impropriety of including the SAT scores by saying:
“As I have not followed the usual path of going to college, I here provide my SAT scores …”?
Bryan wrote:
“In equilibrium, then, we’d expect workers to do employers’ dirty
work for them. Since firms risk lawsuits if they ask for test scores,
applicants would volunteer their test scores without being asked.* When
workers don’t deliver their scores, would employers quietly assume the
worst and hire accordingly.”Does this slight edit change the signaling?
In equilibrium, then, we’d expect workers to do employers’ dirty work for them. Since firms risk lawsuits if they ask for Facebook passwords, applicants would volunteer their Facebook passwords without being asked.* When workers don’t deliver their Facebook passwords, would employers quietly assume the worst and hire accordingly.
Interesting turnaround, but is it relevant? Are you saying that applicants don’t share their test scores because they feel it’s a massive invasion of their privacy? Doesn’t sound plausible to me.
Why is it “just not done”? I think because it breaches some sort of
signaling manners. You’re supposed to signal how intelligent you are,
but not quite so directly and blatantly, which may come across as
announcing that one is better looking than 85% of other people.
My immediate response to this seems to be no one’s else’s response: why
would you believe them? Schools get official ETS score reports.
Employers would get someone volunteering their test scores with no way
to confirm this.
A partial solution would be to attach a copy of your test results. Yes, it could be forged, but that’s a lot harder than baldly lying.
If I were a signaling skeptic, I’d look at all these results and say, “The signaling model is meaningless. Almost anything can signal anything, apparently.” But as Thaler says, I’d rather be vaguely right than precisely wrong. How on earth would a naive human capital story even begin to explain the first-hand experiences readers shared in the comments?
READER COMMENTS
Rohan
May 13 2012 at 10:05pm
I wonder if it is in the employee’s best interest to make it harder for employers to do a direct comparison.
Start from the premise that there is always someone “better” than you out there. If people provide test scores, you provide employers with an easy way of rejecting you. They simply pick the person with the higher test score.
Whereas employment history and skillset is more unique. Very few people have identical employment histories. That means that the employer has to do a more thorough comparison and actually evaluate the resume.
So maybe people do not put scores on resumes out of fear that they will lose out to someone better. The only people without that fear are the people with perfect scores, and those people generally don’t need to put their scores down, the regular resume is enough for them to get the job.
An interesting parallel, in a more trivial environment, is the use of mods like Gearscore in online games like World of Warcraft to determine ad hoc group composition.
tim
May 13 2012 at 10:21pm
I interview at least one person a week. We ignore all scores and I find it odd when they show up on a resume (one person put down their HS GPA – really how relevant is that?). Hell – I ignore degrees in most cases (unless its a management position).
Our entire interview process is based on what the candidate knows and how they can execute.
Andy
May 14 2012 at 1:55am
Isn’t there too much risk of lying if applicants put their scores on resumes? When I see a resume I basically assume everything has been manipulated except for actual places of education or employment, because those get verified with a background check before we actually hire anyone. But we wouldn’t really be able to verify your “140 IQ score”, so it’s fairly meaningless.
Ghost of Christmas Past
May 14 2012 at 3:50am
I think it’s a nearly bistable system where one of the stable states is unreachable due to some inessential but powerful influence.
We can’t get to the state where test scores are routinely offered and considered because of the EEOC etc– not even by candidates volunteering scores, because if any large fraction did, then for all the EEOC or private litigants could tell employers might be considering those scores, which would have “disparate racial impact,” and would therefore be unlawful.
(In theory a few people providing scores could “start a trend” which, if encouraged by employers, might “tip” the system toward the state where everyone provides scores, but as we have already seen, employers dare not encourage any such trend so the tipping can never occur.)
I think this is one reason, though probably not the largest, why employers disdain scores: if word got around that they valued scores, more applicants would offer them, and the racial-spoils police would be right on their heels.
Anyway, employer skittishness drives applicants away from providing scores and the other stable state is the one in which very few applicants provide them. People who do provide them, when most people don’t, are by definition “odd” and for most jobs “oddness” is a negative qualification. (Of course as others have pointed out there are exceptional fields or exceptional hiring managers.)
Under this regime, applicants are most tempted to advertise their scores when those are very high, high enough that their prestige may justify the oddness of mentioning them. However, this is a socially-dangerous business. The higher the applicant’s scores, the more likely that they are higher than the resume-reader’s own scores! Very many hiring managers (or just HR dweebs, at big firms) will feel resentful if they see such high scores, and reasons to avoid hiring their possessor will come readily to their minds (including such unquantifiable assessments as “applicant seems arrogant, a poor fit for our organizational culture”).
By contrast, qualifications presented in narrative form– commonly claims of worthy previous achievements (“increased profits in my division 0.5% by designing and leading paperclip conservation and reuse program”)– do not provoke much resentment. I firmly believe this is because hiring managers faced with such claims can easily imagine themselves to be as or more clever than the applicant: “sure he saved a lot of paper clips! That’s just what I would have done in his place. This kid is a go-getter after my own heart.”
Everyone is good at projecting himself into another’s narrative so long as there are no hard numbers involved. The one thing likely to wake a manager from the pleasant illusion that he is smarter than any given candidate is to see test scores showing the opposite.
Everyone in our culture loves to meet people with lower scores and resents meeting people with higher scores because it is more pleasant to scorn than be scorned. The widely publicized falsehood that intelligence, and therefore test scores, can be increased by studying hard makes this resentment worse, for people are more willing to accept differences they ascribe to luck (“I could have been a great basketball player if I were 6’10” instead of 5’10″”) than to effort. The segment of our culture which produces our managerial class exhorts its children to effort and teaches them to feel shame, leading to resentment, if their efforts seem lackluster. Few managers want to associate with someone whose very presence induces them to feel ashamed.
If we could (abolish “disparate impact liability” and…) move to the state in which everyone would reveal their scores, I think that would be sustainable despite the resentment issue, because of two factors: for many applicants, their scores would not exceed their prospective bosses’ scores; and familiarity breeds contempt– confronted by many examples of candidates with high scores, bosses would soon lose their fear of them. Managers would see that many people with high scores are not more successful in daily work than colleagues with slightly less exalted scores, and bosses would soon persuade themselves that the bosses’ own socially-dominant qualities of arrogance and so-forth are more worthwhile than high test scores “anyway,” at least for rivalling the boss, which is the main point of concern. Bosses would end up glad to hire more productive (because more intelligent) subordinates once they learned that they could do so without endangering their own positions.
Martin
May 14 2012 at 6:04am
Bryan,
It seems to me that you’re assuming that test scores would be the only signal send, what other signals do applicants usually send with their resume?
Let’s say that there are two possible signals: school grades and test scores. Where school grades are the result of a battery of tests that correlate well with one another over a number of years and test scores are a one time score that correlates reasonably well with that performance. Both school grades and test scores correlate well with job performance. Both signals are a result of ability and effort such that score = a*effort + b* ability + epsilon. It’s assumed that a is roughly the same for both tests, but that b on average is the same per school test as for the test score.
I.e. when ability differs per school test it’s effort that has to make up.
If all applicants already volunteer school grades, what type of candidate will volunteer test scores?
I) If the test score signal confirms the school grade, what information will the applicant send to the recruiter when the test score is included?
II) If the test score signal ‘falsifies’ the school grade, what information will the applicant send to the recruiter when the test score is included?
I am guessing that for “II” it shows your private cost of effort, meaning that for “I” it’s pointless. For high ability, you know that the cost of effort is low with “I”, or for low ability you will know that the cost of effort is low with “I”. For “II” you will know that the cost of effort is high when test score > school grades.
And what firm wants to hire someone who has many outside activities that he/she would have to give up conveniently labeled “preference for leisure”?
Dan
May 14 2012 at 8:18am
Applicants could just work short-term for a test prep company that requires a high minimum score. By putting such a job on your resume, you signal all at once your intelligence, your ability to communicate indirectly (an important workplace skill), and your ability to work with people.
Dan Carroll
May 14 2012 at 11:41am
Ghost makes an interesting observation. I have another anecdote. While the investment industry does value test scores (which can be verified, if the applicant offers copies of the ETS reports, and if consistent with other aspects of the resume), another oddity does arise: investment performance. It is very difficult to advertise investment success on one’s resume, despite the fact that investment firms and investment professionals live or die on investment performance. Resumes typically get rejected immediately if there are numbers. It is the ultimate IQ test for the industry, such that it could mean big future profits if repeated. Yet, it often creates a visceral negative reaction, often accompanied by disbelief, in the hiring manager.
For that reason, I have very rarely given specific performance numbers, though the subject sometimes comes up during the hiring process. When it does, it is a minefield.
I think there are three reasons: One, the ego effect described by Ghost. Most investment professionals believe they are the smartest guys in the room. They are not threatened by test scores (their own test scores are usually high), but they are threatened by the investment success of others. Two, individuals with good performance will likely expect to be given responsibility (if not in year one, then in year two), and that is not often practical (am I really going to fire or demote the guy who is doing that job now?). Three, how can one be sure it will be repeated?
The key in signaling then may be to find the right balance between being highly qualified without being overqualified. Factor that into the fact that the hiring process in US businesses is actually less effective than throwing darts.
Troy Camplin
May 14 2012 at 1:06pm
Given my inability o get a job with my Ph.D. in the humanities (since I graduated in 2004), perhaps I should submit my GRE scores.
610 – Linguistic
640 – Quantitative
750 – Qualitative (putting me well into the 99th percentile)
Now, who wants to hire me? 😉
Jacob Lyles
May 14 2012 at 9:49pm
Google asks for test scores
KnowPD
May 15 2012 at 9:52am
If screeners believe in human capital theory (strong if not overwhelming bias in business at on all levels based on 10+year experience in business), then even if applicants include IQ/SAT/etc. and it’s an effective signal, the screener isn’t looking for the signal because he’s irrationally biased against it, you wouldn’t see it in the data. Put differently, the bias in business toward human capital theory is so strong that the signaling hypothesis is beyond the realm of “reasonable” consideration so you wouldn’t see it in the data no matter how closely you looked. You’d need a study with data demonstrating that hiring decision makers are biased against their own interests. If you were clever, you could get data to support this hypothesis.
Comments are closed.