As a teacher of high school statistics, I found the controversy over exit polling to be somewhat amusing.
Exit polling is necessarily stratified. You have to plan ahead of time which precincts to sample and how many voters in each precinct to sample. This plan produces a stratified sample by definition.
To use an exit poll to predict the outcome of an election, you need to adjust the weights on your stratified sample…the only information that an exit poll can provide is an estimate of the percent of people in each precinct who voted for each candidate. The exit poll cannot tell you how many voters there were in each precinct.
After the polls close, if the election officials tell you how many people voted in each precinct, then you can use exit polls to accurately predict the outcome of the election.
The essay includes an interactive illustration of the uselessness of exit poll results without weighting information.
What I suspect happened is that unweighted results were released, which is total garbage. Imagine if you surveyed 50 voters each from ten precincts and released the results of that survey, notwithstanding the fact that some precincts had ten times as many voters as others. You could have a totally unbiased exit poll with zero sampling error, but until the stratification is accounted for, you have no information.
For Discussion. If you’re not a statistician or a pollster, could it actually be a good thing for the public to have a myth that exit polling is erroneous?
READER COMMENTS
cb
Nov 5 2004 at 11:19pm
I agree, but it’s arguable. Say you have two polling spots. In the end spot A has 500 hundred votes and B has 1000. It’s not particularly unreasonable to assume that after the first 6 hours, spot B will have more people coming out the door for you to interview than spot A.
Personally, I think there was another factor, people on the right don’t like to be interviewed. I ignored a guy that was trying to get me to sign something that I actually supported, and I would have ignored a exit-poll person also. My position is that I voted, leave me alone. Granted, I tend to be libertarian and I voted in a precinct that was 2-1, at least, against who I voted for (I didn’t want to talk to any of those people), not scientific at all.
There’s also the conspiracy theory, big media is clearly biased to the left, and by ‘conviently’ releasing polls showing a Kerry win discourages the right, blah, blah, blah. I remember seeing the Dow drop 150 points in an hour and a half when those polls hit the net, which had never been released by the insiders before. ‘interesting’
Networks want to deliver news, and they know people would rather not stay up till 5, I’m sure they’re finding out went wrong and will try to correct it, but I do agree that there are obviously some sampling errors. (My boss says that sampling people voting during the day is just asking all the unemployed, ouch.)
Lawrance George Lux
Nov 6 2004 at 2:09pm
Any Information gathering verhicle can be a two-edged sword, and there are Those who each use their own desired way. George W. Bush claims a clear mandate, because of three million more Votes than his opponent. The fact exists that almost 65% of Americans oppose continued presence of American troops in Iraq, Bush has zero support for any military action against Iran, a majority of Americans oppose his Tax policies, SS privatization program, oppose Tax breaks for Corporations (remember the last Corporate Tax relief Bill signed short days ago), and want the current Supreme Court policy on Abortion to be maintained–exactly what Bush has publicly and privately proclaimed would be his second term agenda. Does George W. have a mandate? Especially when he plans to increase Federal spending another 15% Second Term, and will do nothing to constrain health care costs? lgl
David Thomson
Nov 6 2004 at 6:43pm
I immediately realized the exit polls were unweighed when hearing about the high number of women in the sample. As matter of fact, I became quite happy because the initial report said that President Bush was down in Ohio and Florida by a mere one percent. Common sense dictated that this meant the President was actually up overall by two to three percent. Thankfully, I was right. But if I could see this clearly, why didn’t the “elites?” What was their problem?
FuriousJon
Nov 6 2004 at 7:22pm
It would be a WONDERFUL thing to discredit exit poll data, especially if you’re attempting to perpetrate electoral fraud. I don’t know where they got their numbers, but a group called BlackBox has statistics showing vast differences between exit poll data and computer-tallied vote counts in the ten closest Presidential races this election. I find it hard to believe, but it certainly makes sense if you’re looking for conspiracy theories.
Brad Hutchings
Nov 6 2004 at 9:39pm
Here’s the problem with exit polls as a statistical proxy for the real vote. You cannot lie to or evade the voting card. People regularly lie to and evade pollsters, even exit pollsters. I don’t know how a pollster would account for this year’s dynamic… In politically mixed crowds, it was very common to hear Democrats recount the latest Michael Moore conspiracy fantasy and for the rest of us to just kinda have that “whatever” look on our faces.
So what is the point of these things in American Presidential politics anyway? While they ought to tell us who is President soon after the polls close in California, what they ended up doing this time is keeping us all up until the cows went out Wednesday morning. See the accounts of how the MSM (minus Fox and NBC) wouldn’t call Ohio at the request of the Kerry campaign because of the exit poll data. Without the exit polls, we’d have waited a few hours after the polls closed and been done with it without controversy. Eeeeek!
Bruce Cleaver
Nov 7 2004 at 8:42am
David Thomson wrote:
” But if I could see this clearly, why didn’t the “elites?” What was their problem?”
Well, it’s the clearest case of confirmation bias I have ever seen. Not only that, the ‘elites’ (I am talking television broadcasters and print prognosticators) are likely innumerate, and the talk of stratification and the like would cause their eyes to roll back into their skulls. But hear this – if the exit polls had come out with an abnormally large Bush lead, then I conjecture the usual suspects would have quickly looked to discredit the exit polls by any means necessary, and would have been able to discern any methodology errors. SO again, confirmation bias plays the largest role.
Boonton
Nov 7 2004 at 10:39am
has a great article on exit polls. When they are done they are not only weighted for turnout but they also adjust for ‘non-responders’. So if men refuse to answer 50% of the time the results of the poll will be adjusted to reflect that fact.
cb
Nov 9 2004 at 1:27pm
The only problem w/ that is that it assumes the adjustments are correct. It is easy to envision a situation where the percentage of non-responders, etc, change over time. exit-polls are samples, as in the case of all samples, sometimes they’re wrong, I don’t see what the big deal is.
Comments are closed.