Pollster.com

Articles and Analysis

 

About That Michigan Exit Poll

Topics: 2008 , ABC , Barack Obama , CNN , Exit Polls , Hillary Clinton , John Edwards

The exit poll conducted for Michigan's Democratic primary, and more specifically, the way it was used to help allocate Michigan's delegates this past weekend, has been the source of controversy over the last 48 hours. I will let others debate whether it is every appropriate to use any survey -- much less an exit poll -- to award delegates. However, amidst all the spin, some pertinent facts about exit polls and their performance this primary season have been confused.

The best example comes from two brief clips from CNN's coverage of the Puerto Rico results yesterday, the first from an interview of Clinton campaign chairman Terry McCauliffe and the second a response by CNN's Bill Schneider that followed soon thereafter:

Here is the gist of McCauliffe's complaint (my transcription):

The one thing I find amazing is, Wolf, is they say, actually, we are going to base some of this on exit poll data. There has not been an exit poll in this campaign -- I can remember standing in New Hampshire on election night saying, "hey Terry, you're going to lose by fifteen points." None of the exit polls have been right, and you're going to use that to take votes away from Hillary Clinton?

And Schneider's response and discussion a few minutes later with CNN's Wolf Blitzer:

Schneider: I can tell you that the exit polls have been pretty accurate in all the primaries so far, including New Hampshire. The exit polls have been very close to the actual result.

Everyone remembers the Waterloo of the polls in New Hampshire. That wasn't the exit poll. That was pre-election polls. The pre-election polls in New Hampshire, those taken before primary day back in January, many of them indicated that Barack Obama would win New Hampshire. In the event, on primary day, he actually lost New Hampshire. The exit poll got that right.

It was the pre-election polls that did not capture the final last minute swing of a lot of women, particularly older women, who had been undecided, they swung at the last minute to Hillary Clinton and that's what put her over the top.

Blitzer: And when we do these exit polls, we have three waves of exit polls, and sometimes the media gets told of only the first wave which may be distorted, may not be precise. It takes three waves to get an accurate assessment of what is actually going on. As a result, some confusion about the accuracy of these exit polls.

So either "none of the exit polls have been right" or the exit polls have been "pretty accurate in all the primaries." Not surprisingly, the whole truth lies somewhere in between.

Let's start with the comment from Wolf Blitzer at the end of the clip. He is right to point out that exit poll results get phoned in by interviewer/reporters in three waves, and that the biggest "errors" have involved early data based on the second wave or on the third wave called in just before the polls close (the networks now keep data "quarantined" and do not release it to network producers and decision desks until after 5:00 p.m.. Eastern time).

The second wave (late afternoon) estimates have often leaked this year, and those results have, more often than not, erred in Barack Obama's favor during the primaries (especially on 2/5 and 3/4 and in Pennsylvania). As I wrote back in March, "the early leaked results overestimated Obama's strength in 18 of 20 states, for an average error of 7 percentage points on the margin."

Assessing the accuracy of the third-wave, "at poll closing" estimates based only on the exit poll interviews is harder, because those numbers rarely leak. What we see more often, at least indirectly, are the estimates used to weight the official exit poll tabulations that appear on network web sites as the polls close. The estimates are usually a blend of the exit poll interview results and an average of pre-election polls (more details on this process here). These results, as extrapolated by our friend Mark Lindeman each primary night, have been far closer to the final results than the early leaked numbers. When I looked at the numbers from 2/5 and 3/4, I found that while big errors in Obama's favor persisted in six states, the errors in the remaining 11 states were small and canceled out (averaging to zero).

With respect to New Hampshire, I have heard rumors that mid-afternoon numbers showed Obama leading Clinton (Chris Matthews appears to say as much in this clip), but nothing as large as the "15 point lead" that McCauliffe claims. As the polls closed, our friend Mark Lindeman extrapolated a 38.3% to 36.9% margin in Obama's favor from the official exit poll tabulations appearing on network web sites. So if anyone told Terry McCaullife on "election night" that Clinton would lose by 15 points, it was not on the basis of an exit poll.

What is misleading about this entire discussion, however, is that the Michigan results relied up by the DNC on Saturday (and analyzed in more detail by Brian Schaffner) were not the second-wave or "at poll closing" estimates of the official count, but rather the results of this question after the tabulations had been weighted to match the official count:

If these had been the candidates on the ballot today, for whom would you have voted in the Democratic presidential primary?

46% - Hillary Clinton
12% - John Edwards
2% - Dennis Kucinich
35% - Barack Obama
1% - Bill Richardson

To be more specific about the weighting: Once the Associated Press reported a final count for Michigan on the evening of January 15, the exit poll analysts reweighted their tabulations so that the size of each poll region (labeled as "Geo Stratum Code") and the candidate vote shares within each of those regions matched the actual result. Thus, the "vote estimate" at the top of this final tabulation produced by Edison/Mitofsky (and posted online by ABC News) shows 56% for Clinton, 4% for Kucinich, less than 1% each for Dodd and Gravel and 39% supporting uncommitted.

Can we rely on the final exit poll data when weighted data "forced" to match actual results? That is the difficult-to-answer question many of us have been been pondering this year. Does weighting the result by the vote preference and turn out eliminate all possible bias with respect to demographics or other attitudes? Perhaps. In this case, however, the correction for statistical bias is right on point. We know that the results are weighted so that the percentage who chose Clinton matches the actual count.

Actually, if anything, the final weights may favor Clinton slightly, for two reasons. First, the truly final count (available after these tabulations were done on the evening of January 15), gives Clinton 55% (not 56%) and undeclared 40% (not 39%).

Second, as reported on Saturday, the official count did not include approximately 30,000 write-in votes that were never counted or included in the official totals because no candidates filed the necessary papers to request the counting of write-in votes. Most assume these write-in votes were cast for either Barack Obama or John Edwards. The voters who cast write-in votes presumably had no idea their write-in votes would not count as they left their polling place and, we can assume, would have been just as likely to participate in the exit poll as other voters.

The exit poll questionnaire had a response option for other ("Other: Who? _______") that, presumably, would have been chosen by write-in voters (though I am not sure how the exit pollsters handled any such responses in the final tabulations). Since no write-in votes were reported, however, the weighting of the final tabulations did not reflect votes that could have increased the total vote by as much as 5%. So the weighting of the exit poll -- like the official count -- may have overstated Clinton's vote by a few percentages points over what it would have shown had all write-in votes been counted.

Another potential source of error would be those voters who cast absentee or early. Michigan allowed for early voting, but in this case, the exit pollsters did not conduct a telephone survey to specifically capture the attitudes of absentee voters. I have not been able to find any report on the percentage that voted early or by absentee ballot. However, the final tallies used to weight the final tabulations included absentees.

Again, reasonable people may question whether it is ever appropriate to use any survey -- no matter how accurate -- to allocate delegates from a primary election. However, the case for labeling this particular application of this survey as inaccurate is weak. The final Michigan exit poll tabulations are best evidence we have on which candidate voters would have favored had the names of all candidates appeared on the Michigan ballot. The weighting procedure provides reassurance that, in this case at least, the percentage of Clinton voters was either right or erred slightly in her favor.

 

Comments
eternaltriangle:

I am fine with using exit poll data, and other stuff, to help address highly unrepresentative processes. I only wish the same logic were applied to caucuses, which have been far more skewed than the Michigan primary.

But seriously, using a survey (if you have a truly random survey, that is not biased - which unfortunately, exit polls are, and they tend to lean Obama, just as they lean Democrat) does not challenge democratic principles.

However, if you had good surveys, it does not contradict fundamental democratic principles.

1. Every vote counts
Just because only some proportion of votes are counted, if everybody has an equal PROBABILITY of being included, then every vote does count equally.

2. Delegates/seats/victory should be based on the will of the people
Firstly, no election rule can satisfies this notion (see Arrow's Impossibility theorem). Using polls induces a margin of error - but then elections surely also have a margin of error, as any resident of Palm Beach will tell you.

3. Polls can be more representative
Because voting requires the expenditure of transaction costs (time), or even more, in the case of caucuses, random surveys are actually more representative of the population at large.

4. Survey-based democracy can involve more frequent input
Because it is cheaper and quicker to use surveys than to use elections, you can have democratic input more often, again improving democracy.

I hardly think surveys run counter to democratic principles and, in fact, they enhance them, over actual elections. As internet use approaches 100%, we approach considerable technological possibilities for citizen engagement.

____________________

Ciccina:

Mark - really, really, really interesting stuff.

Eternaltriangle... I know you're probably right; the math is more than I can handle at the moment. But surely there is something fundamental about making a good faith effort -- trying really hard -- to physically count every vote.

I think its outrageous that 8 years after that debacle in Florida, we still read things like, "the official count did not include approximately 30,000 write-in votes that were never counted or included in the official totals because no candidates filed the necessary papers to request the counting of write-in votes." What the ? Every vote needs to be counted no matter what the candidates file, no ifs ands or buts. And if the boards of election really can't handle it, then don't offer the write-in option. Jeebus! Its disgraceful.

I wonder, was there anything else on the ballot that MI voters could have voted for? Because I'd be interested in knowing if there was a discrepancy between the total number of votes cast for the down-ballot item vs. the total number cast for presidential nominee. Just a thought.

____________________

patrick:

One fundamental point that's not been mentioned here is that in the history of presidential primaries--going back to 1912--no candidate ever has been awarded any delegates through the primary process if he was not on the ballot. In this case, four candidates removed their names from the ballot on their own volition. It is passing strange, then, that exit polls were substituted for votes cast, regardless of issues about the accuracy of the specific exit polls. Polls were not designed for the purpose of substituting for votes.

____________________

cinnamonape:

patrick:

The Michigan case was clearly unusual and a compromise in obtaining n understanding the "will of the voters" in a Primary which was discredited as "not counting" (even by Clinton). In fact, as pointed out by the DNC legal eagles...the Rules Committee could have denied the entire delegation participation.The settled on the minimum of the mandated penalty -which was half votes per delegate.

It's clear that a large proportion of the Democratic voters failed to participate in the election (in a majority Dem. State it was the only primary that drew less than the opposite party in a season in which Democratic voters usually outnumbered the Republican turnout 2:1)

...and most of these potential "lost" votes (though not all) would have been for candidates that had removed their names from the ballot. There is no way to determine who these stay-at-homes would have voted for...even from an exit poll.

Thus any allocation of votes in Michigan is likely to be biased toward Clinton, who was ON the ballot. But awarding her 100% of the delegates smells like something that occurs in Zimbabwe.

This immediately then raises an issue of the allocation of the votes "within" the election. Even write-ins were not allocated (50,000+ for Obama). One could have given the votes to Clinton that she received...but what to do with the "Undeclared" bloc. One could give it entirely over to the sole individual remaining in the race who was "not Clinton". But that would likely be unfair, as individuals who supported Edwards may not necc. consider Obama as second choice. That leaves the only other solution as one in which surveys (very imperfect and varied) of the exit polls are used.

"Undeclared" delegates are allowed to vote for their personal option at the convention...but are clearly intended to be individuals that sue the process during the campaign to make their decisions. Once again "how" does one select actual "Undeclared" delegates.

A third option, only allowing Michigan delegates after the initial convention ballot to vote on subsequent ballots was an option (and could have been applied w/ Florida, as well)...then all delegates are "free". But this would have resulted in Obama winning on the first ballot, most likely, and there never being a second round of voting...other than an "affirmation."

____________________

AJbaker:

Michigan state law requires the candidate to give permission before any write in votes for that candidate can be counted.

There have been a lot of comparisons of one candidate elections to Russia. However, it is clear that in a true democracy once a candidate takes his name off the ballot votes for him are not counted. If your candidate takes her name off the ballot who should get the blame for the fact you can't vote for him.

As of the night before Deb Dingell who had pled with Obama to stay on the ballot believed that she had convinced him.

Obama first claimed that he was required by the rules to take his name off, and then claimed that he was doing so "in deference" to the rules. Actually, he was under pressure from New Hampshire and successfully curried favor in Iowa. It was a good tactic but now he has a strategic problem trying to mollify offended Michigan voters such as myself.

____________________

CaptainPatent:

In all honesty I don't think there was a truly fair way of handling Michigan at all after it was declared void.

It's true that Obama unfairly received uncommitted votes that were meant for Edwards and Richardson. It's true that Clinton received both "oh well" and protest votes because of the candidates who took their names off the ballot. It's true that either candidate could have potentially faired much better if campaigning would have occurred and the entire public who wished to vote actually came out to vote.

Fingers can be pointed in any direction (and they have been), but no matter what conclusion or mathematical formula you apply it is educated speculation at best.

I don't want to sound like I think the compromise was a horrible one, because I don't think it was, but the DNC really needs to stick with a punishment... and probably only give the minimum punishment unless the state committee has repeatedly violated a given rule.

One more note - I watched the full 6 hours of meeting coverage and it was painful to see how split portions of the democratic party were even though both solutions (Obama and Clinton) are very similar.

I just hope the democratic party can heal from this blow, and I at least will still be voting for either Clinton or Obama in November.

____________________

boskop:

honestly, i hope the democratic party doesnt recover,they dont deserve to. i have oft stated that they hardly deserve their moniker given their emphasis on rules over principle. had they determined from the start that 50 states had the moral imperative to vote at least in this so called 'democratic' party, then the rules should have been altered to embrace the inevitable glitches that will always come up.

such is life. but principles must remain hard and fast beyond the vicissitudes of political strife. i can see republicans pulling such a dogmatic and autocratic stunt. but now, having been partial to the democratic party for so long, i big it adieu and will not look back on Gomorrah.

sadly, at its moment of greatest symbolic impact, it was unable to support its pledge
of equanimity. now my only hope is that mccain listens to the slimy likes of dick morris who actually had wise words for him: go back to your real identity john mccain as the only real
uniter, shift left not right.

if he understands that many such as myself feeling as i do are ready to jump, i caution him to speed it up. let the democrats burn themselves up as they hurtle forward on this
disastrous and preposterous scheme to anoint
the most adept political hack of recent times.

and for all those who hurl 'racist' at our likes i suggest that it is racist to elect someone for the sake of historical narrative. you are doing it because he is black. were he not, he wouldn't have made it out of the gate.

that is playing the race card. i am looking at the man. he is not sufficient.

caveat emptor.

____________________

CaptainPatent:

@boskop

"but principles must remain hard and fast beyond the vicissitudes of political strife. i can see republicans pulling such a dogmatic and autocratic stunt. but now, having been partial to the democratic party for so long, i big it adieu and will not look back on Gomorrah."

You'll have to clarify this statement a bit because as phrased, it sounds like you're stating that the Michigan voting debacle is something that would happen to Republicans and would be an accepted norm... but because it happened to Democrats you switched parties.

"sadly, at its moment of greatest symbolic impact, it was unable to support its pledge
of equanimity. now my only hope is that mccain listens to the slimy likes of dick morris who actually had wise words for him: go back to your real identity john mccain as the only real
uniter, shift left not right."

This is another statement I find strange. Yes McCain is very Moderate, but why would you vote "hop[ing] that McCain [...] shifts left not right" instead of voting left and leaving speculation out. I don't see how the decision of a couple of committee members with little foresight can speak for a whole party which is what you seem to imply.

"if he understands that many such as myself feeling as i do are ready to jump, i caution him to speed it up. let the democrats burn themselves up as they hurtle forward on this
disastrous and preposterous scheme to anoint
the most adept political hack of recent times."

I don't quite understand what "scheme" you're referring to. You'll have to clarify a bit. I also don't quite know what your qualifications are for being a "hack."

"and for all those who hurl 'racist' at our likes i suggest that it is racist to elect someone for the sake of historical narrative. you are doing it because he is black. were he not, he wouldn't have made it out of the gate."

Am I? Well it's great you could clear up intent for every democratic voter in that blanket statement there.

I will give credit where it is due though: There are people who are voting specifically for historical narrative which is sad. After seeing interviews (especially, but not limited to, after Kentucky and West Virginia) it was also apparent that people are also voting away for racial reasons. I'm not going to speculate as to the reasons you or anyone else is voting in the manner they vote, but I only hope race ends up being an insignificant factor. I don't think that will be the case in this election, but I hope that does happen.

"i am looking at the man. he is not sufficient."

Just as a side note, did you initially lend support to Hillary Clinton or have you backed McCain through the whole process? Only curious.

____________________

patrick:

Yes, cinamonape, I know all the talking points on this. But there is a long history here, and historically candidates who take their names off ballots don't get convention delegates. But more germane to this thread, it's a misuse of polls to use them in place of votes to allocate delegates to a national convention. The problem perhaps would be more obvious if this were a direct election, and an exit poll were used to determine the winner. There is some danger that an unfortunate precedent has been set here.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR