Pollster.com

Articles and Analysis

 

Exit Polls: What You Should Know 2006

Topics: Exit Polls

Yes, the television networks will be conducting exit polls today. But if you are looking for the leaked exit poll estimates that typically appear online on Election Day, you are probably out of luck at least until later tonight. More on that below. But as long as you are here, let me tell you a little bit about how exit polls are conducted, how they will be different this year, and why it is probably best to try to ignore the exit poll estimates that will inevitably leak later tonight.

I have always been a fan of exit polls. Despite their shortcomings and the inevitable controversies, the final network exit polls remain our best source of data on who voted and why. Having said that, exit polls are still just random sample surveys, possessing the usual limitations plus some that are unique to exit polling.

A quick summary of how exit polls work: The exit pollster begins by drawing a random sample of precincts within a state, selected so that the odds of any precinct being selected are proportionate to the number that typically vote in that precinct. The National Election Pool (NEP) consortium, which is conducting the exit polling for the six major networks, will send exit pollsters to more than 1,000 precincts across the country today.

One interviewer will typically report to each sampled precinct. Each interviewer will stand outside and attempt to randomly select roughly 100 voters during the day as they exit from voting. The interviewer will accomplish this task by counting voters as they leave the polling place and selecting every voter at a specific interval (every 3rd or 5th voter, for example). The interval is chosen so that approximately 100 interviews will be spread evenly over the course of the day.  As we learned following 2004, random selection of voters at the polling place is the most important part of the process, and arguably the most susceptible to problems.

When a voter refuses to participate, the interviewer is instructed to record their gender, race and approximate age. These data allow the exit pollsters to do statistical corrections for the bias in gender, race and age that might result from refusals.

The selected voters receive a one-page paper questionnaire to fill out. In the past, the questionnaire included approximately 25 questions (see an example from the 2004 exit polls), although this year the exit pollsters have worked to prepare a shorter questionnaire. Respondents fill out the survey privately then place it in a clearly marked "ballot box" so they know their identities cannot be tracked and their answers remain confidential.

The logistics of transmit all the results to a central location quickly and accurately provides the biggest challenge. To facilitate the process, interviewers will take a 10 minute break during the day to tabulate responses. Interviewers have typically stopped to call in their tabulations at three approximate times during the day: 9:00 a.m., 3:00 p.m. and shortly before the polls close.

Once the polls close, the interviewer will attempt to obtain actual turnout counts, and if possible, actual vote returns for their precinct from polling place officials. One of the unique aspects of the NEP exit poll design is the way it gradually incorporates real turnout and vote data as it becomes available once the polls close. The exit pollsters have developed weighting schemes and algorithms to allow all sorts of comparisons to historical data that supports the networks as they decide whether to "call" a race for a particular candidate. When all of the votes have been counted, the exit poll is weighted by the vote to match the actual result.

Throughout the day and after the polls close, the exit polls are a part of a larger process that provides network analysts with various "estimates" of the vote in each state. They have esoteric names like "Best Survey Estimate" and "Composite Estimate" (a post-election report on the 2004 exit polls had some rare definitions and examples, see p. 7). Before the polls close, those estimates derive mostly from exit poll data, but as actual returns become available, the computer models gradually replace the exit poll tallies in each precinct with hard vote counts.

After the polls close, the network analysts also begin to look at estimates based on larger samples of approximately 2,000 randomly selected precincts where NEP personnel obtain official vote tallies as they become available. The networks typically use this larger system of tabulations and estimates -- rather than the exit polls alone -- to call close contests after the polls close. But when the vote is really close, as it is likely to be tonight in states like Missouri and Virginia, even the larger samples of hard returns will be inadequate. They will have to watch the complete vote count, just like the rest of us.

In years past, hundreds of producers, editors and reporters would have access to the mid-day estimates starting at about noon, and preliminary exit poll cross-tabulations (like this one) would inevitably leak out.. This year the networks have taken steps to prevent such leaks, creating a "Quarantine Room" where, as the Washington Post's Howard Kurtz reports, "two people from each of the networks and the Associated Press" will be allowed "entree to a windowless room in New York [where] all cellphones, laptops and BlackBerrys will be confiscated. The designated staffers will pore over the exit polls but will not be allowed to communicate with their offices until 5 p.m."

So there you have it folks. At 5:00 p.m. the data will be flowing to newsrooms as it always does and the usual leaks will presumably commence shortly thereafter. One rationale for the delay is that the data available at that hour will be more representative and not just based on those who vote in the morning.

So if this process is so sophisticated and the late afternoon numbers are better, why can't we rely on the leaked "numbers" that will undoubtedly leak after 5:00 p.m. today?

1) It is still just a survey -- Even when complete, an exit poll still has the same random variation as any other survey. NEP says typical state exit polls have a sampling error when complete of 4% at a 95% confidence level, and 3% for the national exit poll.

But that is the margin of error recommended for reading the cross-tabulations. The exit networks pollsters require far more statistical confidence before projecting winners. As Joe Lenski, whose company directs the exit poll operation, told me in an interview last week, the "rough rule of thumb" is to require "three standard errors, which would be 99.5% confidence" before projecting a winner. While the precise numbers will depend on the circumstances, that implies a much larger "margin of error."

2) Early or absentee voting -- The ABC News/Washington Post survey released on Sunday reported that roughly 12% of all registered voters nationally had already cast absentee or early ballots. Obviously, these voters will not be available to interviewers standing outside polling places. To incorporate early voting, the National Election Pool is doing telephone interviewing in many states to sample the votes of those who voted early (although they will not do telephone interviews in Maryland, where doubts about electronic voting have caused absentee requests to skyrocket). Will these results distributed to NEP subscribers after 5:00 p.m. include the telephone results for early voters? Who knows? I wouldn't count on it.

3) Past Errors favored Democrats -- In past elections, the results have slightly favored Democratic candidates. As we learned after the 2004 elections, the exit poll estimates have had at least some small error favoring the Democratic candidate for president in every election since at least 1988. In 1992 and 2004, the two elections with the highest levels of voter interest and turnout, those errors were more acute.

Since 2004, the exit pollsters have taken steps to try to eliminate those problems. As Joe Lenski explained, they have shortened the questionnaire, improved interviewer training and made an effort to hire older interviewers (because of problems that younger interviewers had in past years gaining cooperation from older voters).

They have also gone to court to allow more interviewers to near polling place exits. Hopefully, these efforts will reduce or eliminate the problems experienced in 2004, but we will have no way of gauging their success at 6:00 p.m. tonight.

4) Limited Data on the House elections -- While the exit polls will produce vote estimates for the major contests for Senate and Governor, they will not make projections on individual House seats. The networks will do that based on actual votes. The national exit poll sample will include an estimate of the national vote for Congress by party, but as Joe Lenski points out, "because of Gerrymandering, we know that popular vote for the House does not translate into House seats directly." (See also Professor Franklin's post on the imperfect relationship between the national vote for Congress and seat gain or loss).

Of course, we are all intensely curious about the outcome of today's elections, and should the exit poll numbers leak again, most of us will find a way to peek at them.  I just want you to know that those leaked exit polls really don't tell us much more about the outcome of the race than the telephone polls we have been happily obsessing over at Pollster.com all fall. Even if we wanted to call a race on unfinished, late-day exit polls alone (something the networks will not do), we would need to see differences of at least 8-10 points separating the candidates to be statistically certain about the outcome.

So look at them if you must, but please, don't go leaping to conclusions based on a leaked exit poll margins showing one candidate "leading" by a few percentage points. Those numbers are not magic. You would be better off flipping a coin -- or better yet, checking the final public poll averages here on Pollster -- to determine the outcomes of the Senate contests in Missouri, Virginia and Montana.

PS: All of the above applies to true "exit polls," those that intercept voters as voters exit the polls. In the vacuum created by the "Quarantine Room," I would not be surprised to see someone release results from a "day of" telephone poll billed as an exit poll. Don't let the name fool you. If the network exit polls are of little value in close races, imagine the potential problems facing a telephone survey, conducted during the day when most voters are still at work. If one of these pops up, I'll be back to blog about it.

I will be online all day. If you have questions about exit polls or anything else, please send them my way.

The following are links to the NEP network web pages that will post exit poll results at some point.  Most of these are either not yet displaying results page or all also include actual vote returns:

You might also want to check some of the following web resources for more information:

Update: I'm live blogging election night in the next post.

 

Comments
Sam:

Mark, in your opinion, what's the best website to follow through the evening for election results? I used DKos and MyDD before, but always had to deal with the inevitable slowness and crashing. What are your favorites?

____________________

Karl:

Mark,
Thank you for your in depth look at the world of polls. I have been an avid reader for the past month. I find your blog invaluable, not just for the complete list of numbers, but for the inventive ways that you slice them up and for the insightful look beyond the numbers that you and your guest contributors provide.

Thanks again,
Karl

____________________

Me Again:

I just noticed that there is no television coverage of the election this year. The networks are planning a single hour at 10 p.m. Eastern and no PBS coverage. Cable might be covering it, but 40% of the USA (me included) do not subscribe to cable TV or have satellite access. And the networks wonder why their ratings are dropping...

____________________

NancyB:

The late great Chicago Tribune columnist Mike Royko suggested that people lie to exit pollsters. I remember the shrieks that ensued.

____________________

Gonzo:

Mr. B.

Do you have an opinion of WHERE the democratic favoring lean in the exit poll results is introduced? Is it young gregarious pollsters attacking like kind non-refusing subjects? Indeed, something as simple as no dress-code for poll takers and things like hair style and body art could leave to "identification" by possible subjects causing some to reject and others to accept the poll in a manner that would be given to conservative / liberal splitting.

Is it that people have become conditioned by the venomous political debates at work to simply "lie" and say they voted democrat because they're worried about having conflict? In my workplace, several colleagues have revealed to me that they "pose" as democrats because their democratic coworkers are really hard on them to the point of threatening friendships, based on political views and that they don't feel that conservatives will judge them so harshly. Could this be happening?

What is it? I'd guess you've given this some thought, and perhaps could link a post or two addressing the topic?

____________________

roux:

I don't like exit polls. There's a reason for the curtain in the booth.

____________________

Mark Blumenthal:

Gonzo,

I've given it a lot of thought and written a lot of posts, but best is probably Part IV of my review of the RFK Jr. article in Rolling Stone.

PS: To those posting comments today. We have "cached" our servers to help handle the expected crush of traffic. So your comments may take 10 minutes or more to appear.

____________________

Dan:

As a rule, I lie to pollsters. I don't believe that it is a good thing when politicians cater to polls. I think that we should vote for people who believe in something and act accordingly. I would prefer to see the pollsters discredited so that they are not relied on by craven hacks.

____________________

dice:

YAWN!!!!!!!!!!!!!!!!!!!!1

____________________

David R. Block:

OK, but let's say that one lives in an early voting state and has already voted. They will not be in any exit poll (because most of the attention is paid to the standard election day).

Are any allowances made for this in polling??

____________________

Mark Lindeman:

David, see Mark B.'s numbered point 2 above. An effort is made to factor in early/absentee voters based on phone-poll data where available.

The pollsters seemed pretty happy with their 2004 telephone interviews (judging from the evaluation report) except that they wanted to conduct a lot more of them in the future.

____________________

Valentin Castellon:

If anything, exit polls can detect irregularities in the voting system, be machine clicth or fraud. Something we as voters can protest the outcome. We live in different times and there are powerful people that want to win with a lot of money to do it. We do need a last resort oversight. That said, the pollsters have to be bias

Valentin Castellon

____________________

Kenneth Matis:

You've stated that the best predictor is to look at the average of all the polls. However, somewhere in the back of my mind, I remember my stat instructor in college telling us to be careful of averaging averages. Do this memory have anything do with what you're doing by averaging the polls?

____________________

alan:

There was a lot of talk in the TN senate contest about race and gender, so I'm studying the exit poll trying to see these factors influenced the result.

The exit poll does not look right. Maybe someone can help me understand.

http://www.cnn.com/ELECTION/2006//pages/results/states/TN/S/01/epolls.0.html

These two sections in particular are what look wrong:

VOTE BY RACE AND GENDER - votes for Corker
Non-White Men (6%) 12%
Non-White Women (9%) 9%

VOTE BY RACE - votes for Corker
African-American (13%) 4%

How it is possible that 12% of the non-white men and 9% of the non-white women voted for Corker, while only 4% of the African-American's voted for him. Latios, Asian and others make up 2% of the voters, or about 13% of the non-white voters, and even if we assume all of them voted for Corker, you can't get those #'s in the crosstabs to reconcile.

Am I reading this right?

____________________

I have always thought that we can count on numbers, don't you?
anyway, I think the absentee voting is a shame!

____________________

I have always thought that we can count on numbers, don't you?
anyway, I think the absentee voting is a shame!

____________________

I have always thought that we can count on numbers, don't you?
anyway, I think the absentee voting is a shame!

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR