Articles and Analysis


February 5 Polls: Four Cautions

Topics: 2008 , ABC/Washington Post , Barack Obama , Blogosphere , CBS , Gallup , Hillary Clinton , John McCain , Likely Voters , National Journal , The 2008 Race

Over the last 48 hours we have had an avalanche of new polls,** and given the discussion both in our comments section and elsewhere across the blogosphere, everyone seems unsure of what to make of the results and what they say about where things stand, especially in the Democratic presidential race. As is evident from our charts, the trends are highly favorable to both John McCain and Barack Obama, but from there things get murkier, especially in the Democratic race. Here is my sense of what the poll results tell us and what they do not.

The Republican race is easier to gauge, largely because of the "winner-take-all" rules that apply in so many Republican primaries. The National Journal's Campaign Tracker shows that more than two-thirds of the Republican delegates up for grabs tomorrow will be awarded on a winner-take-all basis either by state or congressional district or some combination of the two. As such, John McCain's roughly twenty-point leads in most of the national surveys, combined with similar margins in the winner-take-all-states in the Northeast (New York, New Jersey, Connecticut and Delaware) and narrower leads elsewhere position him to take a commanding delegate lead tomorrow night. Mitt Romney's hopes, on the other hand, ride on surpassing McCain in states like California and Missouri.

The Democratic contest is obviously much closer, although in some ways the process of selecting delegates is more straightforward. The allotment of delegates is proportional to votes in each congressional district and each state. While the rules may make for some odd outcomes in individual states (see more detailed explanations here and here), the allotment across all states should be a good reflection of the overall votes cast. While winning individual states may have symbolic value in terms of the way the media covers the results, the total delegate counts amassed across all states are what really matter.

So what do the polls tell us about how tomorrow's Democratic contest will translate into delegates? While Barack Obama appears to be gaining support, there are four reasons to be cautious about what the various polls are reporting [about where the race will end up -- see the clarification below]:

1) Polls are of little use in the caucus states. Roughly 13% of the Democratic delegates chosen tomorrow are from six states and one territory that hold party caucuses (Alaska, Colorado, Idaho, Kansas, Minnesota, North Dakota and American Samoa). Accurate polling in these contests is next to impossible because past turnout has been so light. Fewer than one percent of the eligible adults in the six states participated in the Democratic caucuses in 2004 (ranging from 0.1% in Alaska to 2.2% in North Dakota).

Turnout in the February 5 caucuses is anyone's guess, and as such, pollsters have wisely stayed away. We have logged only two polls in the six caucus states fielded since December. One of these was the Minnesota Public Radio News/Humphrey Institute poll (pdf) that explicitly warned it was "not a prediction of Tuesday night's precinct caucuses" because "the interviews did not identify likely caucus participants." The second, a Mason-Dixon survey in Colorado, is now nearly two weeks old and gave no indication what percentage of Colorado adults were deemed "likely caucus goers."

2) National polls may be misleading. Given the proportional allotment of delegates across such a large number of states, the national polls may provide a reasonable assessment of where the race stands. While we have a lot of very recent national polling data showing Barack Obama gaining, we have to remember that the February 5 states may look different than those not holding contests tomorrow.

So far, I have seen only two national surveys attempt to break out results for the February 5 states, and those show contradictory results.

The report released yesterday by the Pew Research Center allows a comparison across their last three surveys of Democrats in the February 5 states to those who will vote in later primaries. In the December and January surveys, Pew showed no significant difference between these two categories of states. Now, however, Obama does slightly (though not quite significantly) better in the February 5 states. Looking at it another way, virtually all of recent Obama's gains on the Pew survey have come from the February 5 states.

02-04 Pew-2-5.png

On the other hand, the new CBS News survey, which shows the national Clinton-Obama contest deadlocked at 41% each for Clinton and Obama, yields the opposite result. The CBS summary reports the following about a similarly small sample of Democratic primary voters:

The picture in the states voting on Super Tuesday is not nearly as close as the overall picture and offers some good news for Clinton. Among voters in those states, she leads Obama, 49 percent to 31 percent, with 16 percent still undecided.

As Josh Marshall points out, the entire CBS survey was based on 491 Democratic primary voters, so the subgroup of February 5 state voters may have been as small as 200 interviews.

Perhaps our friends at Gallup, who have interviewed nearly 2,200 Democrats over the last five days, can run a tabulation that helps clarify how the February 5 states compare to the rest of the nation. [Update: They did just that -- details here].

3) Are they sampling truly "likely voters?" Some national surveys, such as ABC/Washington Post and CBS, have reported the results of respondents who describe themselves as likely primary voters. Others, however, have reported on the views of registered voters or adults that identify as Democrats. While turnout is likely to be higher tomorrow than in 2004, the percentage of adults that vote in the Democratic primaries is still likely to be smaller than the percentage represented by most of these national surveys.

Here are two sets of turnout statistics to chew over. First, consider how turnout has increased in the Democratic contests held so far:

02-04 turnout so far.png

As should be obvious, turnout has increased dramatically in all the early states even (or perhaps especially) in states that featured little or no active campaigning by the candidates. If nothing else, this pattern suggests that turnout will exceed 2004 levels in all the February 5 states.

It is still worth considering that past turnout has amounted to a relatively small percentage of eligible adults in each state. The following table shows the turnout levels from 2004 for the February 5 primary states as a percentage of all adults an of eligible adults (as reported by Michael McDonald):

02-04 turnout 2004.png

Here's the main point: Even if Democratic turnout doubles tomorrow as compared to 2004, the percentage of adults participating in the Democratic primaries will still be a fraction of the adults identified as Democrats or Democratic "primary voters" on most national polls. Do the truly "likely" voters look different than all Democratic identifiers? Are the statewide surveys doing a better job of selecting "likely voters" than the national polls? Unfortunately, we can only guess, as only a small handful of the statewide surveys report the percentage of adults that their likely voter samples represent.

4) Uncertainty remains high. If you pay attention to nothing else, remember this: As in New Hampshire, a lot of Democrats are having a hard time deciding between Hillary Clinton and Barack Obama. According to the Pew Research Center, both candidates now receive overwhelmingly positive ratings from Democrats:

  • Clinton: 80% favorable, 15% unfavorable
  • Obama: 76% favorable, 16% unfavorable

Again, as in New Hampshire, voters are expressing considerable uncertainty. In California, for example, both the Mason-Dixon and Rasmussen surveys report 29% of Democrats as either completely undecided or indicating there is still a chance they could change their mind about their preference.

This high degree of uncertainty creates the potential for a volatility that the final tracking polls may not reveal. Many voters will likely carry their sense of indecision into the voting booth, so the news and events of the next 24 hours could prove crucial.

Update: Adam's question in the comments suggests the need for a clarification. I have no doubt that support for Barack Obama has been increasing steadily over the last week. Virtually all of the surveys in all of the states are showing evidence of that trend, and as each pollster measures the same population (however it is defined), those trends are reliable. What I am urging caution about is where Clinton-Obama contest ends up when votes are cast tomorrow. As my AAPOR colleague, Professor Robert Shapiro put it over the weekend, "I would trust the trends but not the magnitude - [it] could be greater or less."

**If you have appreciated the constant flow of updates over the weekend, please post a thank you to the indefatigable Eric Dienstfrey for his exceptionally hard work (and for putting up with a boss who sometimes misspells his name).

Correction: The original version of this post incorrectly identified the CBS News survey as a CBS/New York Times survey.



The numbers that jumped out at me were these:

* Clinton: 80% favorable, 15% unfavorable
* Obama: 76% favorable, 16% unfavorable

I would never have assumed Clinton's favorables were this high given the national press, especially the drumbeat of columnists at the NY Times (Dowd/Rich/Collins/Kristol/Kristof), Slate, Salon, HuffPo, Politico, Atlantic, Vanity Fair, etc. etc.

It seems that voters - or at least Democrats - aren't taking their cues from those we are used to calling "opinion leaders." Maybe their influence is felt elsewhere in the electorate.

Thanks, Mark, for this informative post and thank you Eric, for doing such a great job in continuing to feed the beast.



Thanks, Eric and Mark, for bringing a little clarity to the madness :-)

I have always thought the 45% approving of Senator Clinton are Democrats, and the 45% not approving of her are Republicans. That fits with her 80% favorable rating among Democrats. And I haven't taken my cue from so-called opinion leaders, just my interpretation of national surveys.



Question for Eric and Mark - don't these polls work via telephone? And more specifically, aren't they therefore biased towards people with landlines, since pollsters (like telemarketers) are prohibited from dialing cell phones?

I guess my question is really this - we've seen young voters (under 30) going in overwhelming numbers to Obama, across all gender and racial lines. Coincidentally, voters under 30 are also the demographic LEAST likely to have a landline, with most using their cellphone for everything. So do the polls correct for this or factor it in? Or is the tightening we see in the polls actually UNDERSTATING Obama's potential vote turnout, due to low polling amongst his strongest demographic, the young?



I like and respect this site, and this post offers a helpful and clear-eyed analysis of the recent polling pre-Super Tuesday.

That said, it seems you've made a pretty good case why we readers should basically ignore polls for the next 24 hours, and probably the remainder of the primary season.

Indeed, the entire polling industry has undergone something of a crisis this election cycle, and barring some major new innovation or insight (or at least better explanations, like yours here, of its limitations), I wonder if it can recover.



Yes, many thanks to the indefatigable Eric for the lightning speed with which he posts new numbers.

As for the HRC fav/unfav numbers, what jumps out in the toplines for the Pew is the change in WJC's numbers. Both Clintons now have 52/42 fav/unfav numbers. In December 2006, a little over a year ago, the former Pres was at 63/33.

In contrast, McCain is now at 53/31 fav/unfav and Obama at 58/30.

If the Dem contest turns into a race for super-delegates those Clinton numbers may be a problem for party leaders.


Mark Blumenthal:

@Chaddog: Unlikely there is much of an effect. You might want to start with this post from last Friday which points to the most recent evidence. For what it's worth, both Gallup and California's Field poll now call and interview some voters on their cell phones.

@Hudson: I think that polling has a path to better performance, and it starts with better disclosure. Primary polling is always challenging, and some of it my be doing a very good job modeling likely voters in the primary states. It is just hard to tell good from bad based on the information available.



Mark -- one counterpoint. If it is indeed that there is some volatility which the polls aren't capturing, is it not a remarkable coincidence that nearly every national poll has measured a positive upward trend for Obama?


RS - I shouldn't have written "voters, or at least Democrats" because what I really meant was Democrats. I just assume that columnists at HuffPo, Slate, Salon, and much of the Times are most copacetic with Democratically-inclined voters, and given that, you'd think Hillary's negatives among Democrats would be higher. Pure conjecture from the perspective of someone who has had to deal with "influentials" in the media from time to time.



nice job!


Mark Blumenthal:

@Adam -- Good question. I think the Obama trend is real, and added a an update above to clarify.



@Mark - interesting data regarding cell-only voters, but BOTH of those studies cited in that article conclude that controlling for age, the voters are similar to the national population.

I guess what my question involves is the "controlling for age" or "adjusting for age" aspect of all of this - sure, cell phone voters as a population may have a demographic consensus political viewpoint that is similar to that of the rest of the nation. But given the similarity between the politics of Obama and Clinton, and the fact that Obama CONSISTENTLY has done vastly better than Clinton in the younger cell-only demographic in all exit polls so far, is it possible to "control for age"?

Furthermore, the Field Poll (which is at 36-34 for Clinton) and Gallup national poll (45-44 Clinton) are the polls that arguably have this race CLOSER than other polls.

I guess I'm just still unconvinced that neglecting to poll cell-only voters isn't a huge mistake in THIS primary, where so much of the support between Clinton and Obama breaks down along age demographics - younger cell-only voters going overwhelmingly to Obama, and older land-line owning voters tending a little more towards Clinton.



I don't understand why anyone would be surprised by the high favorability numbers for Hillary. Almost all democrats, even Obama supporters like myself, think highly of her. It's independents and moderate republicans who have disproportionately (and often unjustifiably) negative views of her. That's what worries me - not democrats.


Yeah, I guess Eric's OK... :-)


tom veil:

Thank you, Eric Diensfrey!



I've noticed that the Reuters/C-SPAN/Zogby polls seem to be favoring Obama. Any reason to believe those are any more or less accurate than the others?


Daniel T:

I am a voter in a Democratic super Tuesday state and I am having a hard time making up my mind! I was a Edwards supporter but now that he has dropped out I don't know what to do. If his name is still on the ballot I may decide to vote for him anyway. On the other hand, I am no Hillary fan and I may vote for Obama simply as a protest against her. Or I may simply leave that vote blank and vote in the local races only.

Why am I revealing all of this? Just to confirm that there are a lot of us Democrats that are really undecided as to how to vote. I know several other Edwards supporters and we are all in the same boat. How I cast my vote is likely something that I will only know when I step in a voting booth.

BTW, I do want say kudos to Obama. His supporters were the only ones that showed up at my door. Hillary just sent trash in the mail.



Do the CA polls include absentee ballots? How can one weigh the fact that Independent (3,000,000 of them in CA) can only vote with the Democrats?



Thank you, Eric ... the information is up to date, accurate, and easy to access ... very helpful for us.



Mark and Eric,

Thank you for all of the posts and updates!


jim miller:


Great job---thank you for the great updates and your hard work!!!


st paul sage:

really cool stuff. thank you everybody. a couple of points:

1. that pre-primary polls are not consistently predictive of results is not a crisis for the polling biz. because we don't know who will vote in primaries, it's almost impossible to poll them effectively. this has always been the case.

2. Nonetheless, polls are helpful to tell us what is going on in the delegate race. I believe that Sen. Clinton had a fairly wide lead a couple of weeks ago in most of the Feb 5 states and that Obama has closed the gap this past week significantly. Every poll shows it and it's veruy valuable info. But if you have a state where the voters are now split 40-40 it could have been absolutely accurate even as one candidate takes the state 55-45 tomorrow.


NH democrat:

Latest stats from the NH democratic primary show that 61 percent of registered voters voted in the primary. You may want to reassess your "Democratic Turnout" logic. There is a difference between "eligible adults" and registered voters.



Mark Blumenthal:

@NH: Not sure you're following my point. I'm not trying to belittle the New Hampshire turnout (which is as high as primary turnout gets) but to point out the challenge that all primaries present to pollsters, especially in other states where turnout is a smaller percentage of adults. Remember, most public polls begin with samples of all adults and screen to identify "likely voters."


Thanks for this because I watched the New Hampshire primary predictions on Cable TV and saw that the political pundits got is dead wrong. Perhaps we should do away with the pundits who read polls (can we take a straw poll on that?) and stick strictly to the odds in Vegas. Not that this did me any good in the Super Bowl. cheers


CK MacLeod:

And a 5th caution would apply to variations on the infamous Bradley-Wilder effect. In the past, of course, this was interpreted as a product of racism, implying that voters pretended to favor the African American candidate when questioned by pollsters, but made a racist choice in the privacy of the polling booth. In a Democratic context, I believe that a Bradley-Wilder effect is as or more likely a result of reverse racism. When questioned by a pollster - as by a liberal friend - some voters may be more likely to give the "idealistic" answer, but, in the privacy of the booth, make the "rational" choice. A Mormonism-related version of the same pattern may play a role, and may already have played a significant role, in the Republican contests. It may be very useful to track the outlines of such a divergence between polls and votes, if any, tomorrow, since some version of it may play an even greater role in November.


This election cycle is different. The people have risen above their demographic to vote for the benefit of the country. We see the rise of Obama and McCain - in spite of predictions. We are in a time of Revolution.




From your lips to Gallup's ears (or perhaps, from your keyboard to their inbox). Today's tracking poll shows a surprising uptick for Hillary to 47-43, and a 49-44 edge among Dems and eligibile independents in the February 5 states. I'm surpised by both numbers - that Hillary is gaining ground, and that she does better in Super Tuesday states - but go figure.


Steve Moore:

We als need to consider early voting. I thought I saw somewhere that according to exit polls, Obama won Florida voters who voted election day, but Clinton built up a big lead from early voters that let her carry the state. I know Illinois and California have significant early voting and those people aren't part of late trends.


CK MacLeod:

Good point, Steve Moore. I've been wondering about the same thing, especially regarding Hillary in CA, and wherever else she may have been able to marry organizational advantages to since-vanished polling leads. Early voting advantages didn't seem to help Giuliani very much in FL. On the other hand, his descent started well before election day, and his earlier lead was also diluted some by the presence on the ballot of other candidates who ended up out-of-play.


Bob in Denver:

Any idea why the South Carolina results were so off? The last polls showed Obama leading by 9, the final was a lead of 28.



actually the RealClearPolitics average showed final Clinton poll average of 26.8, and actual of 26.5. it also showed edwards with poll average of 19.2 and actual of 17.6. so the pollsters were pretty much dead-on for those two. my guess is that all the undecideds went to obama last minute. if you look at most of those polls (for all states), they have like 15-20% undecideds. Undecideds broke for Hillary in New Hampshire, and broke for Obama in South Carolina.



Thank you so much for posting this (and for dispelling the myth that the Democratic system of allocating votes is "impossible to understand"). The breakdown was incredibly helpful. We'll see what happens tomorrow!



For the Edwards people that considered casting a vote for him even though he is out, I would submit that you have an opportunity now to pick the candidate that can best compete in the general election and that should be your aim. If you truly want a DEM in the WH in 08, you should think about. For me, I think Obama has the best shot because he is energizing people that have never voted before and bring them into the process. He also has attracted more Independents than Hillary which is good for the general election. Please consider this.


Natalie Rosen:

This is SOOO difficult, I simply am stymied. Of course I will still vote but I am just dying about making the right choice. I don't know I simply don't know. I feel like Obama has what I wish Hillary had -- likability and charisma (although I personally like her a lot). I wish Obama had the precise experience with the proclivity for detail that Hillary has. It is true though Lincoln had no experience when he assumed the presidency. I do though feel like if I voted for Obama though I am betraying Hillary. I LOVE her presentation and she has become such a razor sharp attractive candidate and I LOATH those JERKS at the Times or anywhere else who bad mouth her. If Kristol wants to bad mouth he should look at our brilliant president (I say that sarcastically of course). If Clinton got our country into the absolute horror show mess and deficit and economic chaos that Bush has perpetrated Kristol would be screaming from the rafters. Don't pay ANY attention to these pundits. They do NOT know any more than we do. They are frauds and simply want to foment controversy to sell their show and their commercials and in the case of conservatives want to secure and keep the power of the ridiculously wealthy entrenched to the detriment of most all of us. So, still Hillary is a wonderful candidate much better than any of her criticism would suggest. Having said that Obama is fantastic. I just plead on bended knee that a Democrat wins the presidency. I wonder why it would not be a slam dunk for them with this insipid fool in office who has killed so so many for nothing. THAT is the question. I need a divine vision to tell me what I should do at the voting booth. I'm stymied for the first time in my life!



Natalie....I love them both also...but..just consider..Obama was very thrown off-track by just Bill Clinton's remarks. And he's democrat! Seasoned analysts..reliable folks and also just the common voters say "The republicans will eat him alive! " I think they are right. We HAVE to take back the whitehouse. I am voting for Hillary - for sure! S




Actually if you look at all of the latest polls done, Obama does better than Clinton vs McCain in a hypothetical matchup. The latest I think showed Clinton winning by 1 point or something, Obama by 6.

Clinton is good for the Democrats base. But Obama is more able to bring out people who otherwise aren't inspired to get involved, as well as more Independents and even some Republicans (even though he has a more liberal senate voting record than Clinton according to that recent report).

So much of the country is already divided against Clinton. She'll have a tougher time against McCain in Nov. Obama can inspire more young people to get involved and come out to vote, which will potentially create a landslide Dem victory, because those voters are also more likely to vote more Dems into the House and Senate, giving Obama a mandate to really get some progressive things done.

Just a few things to consider. I understand it's a hard choice for a lot of Democrats to make. I'd be curious to hear what your eventual decision was/is.

Best wishes



[Sorry the above comment was meant for not only Sue but also Natalie]

Final point I forgot to make. "The Republicans will eat him alive" I think is not true. You have to look at who is saying that.. It's more Republicans than anyone else. They would rather battle Clinton because they're used to it. Obama, for all the reasons I gave above and more, will present a fresh new challenge. Obviously they'll throw everything they've got at him, but he has a positive message to repond with, that resonates with people's intelligence. They're watering at their mouths to be able to debate Clinton. How is Clinton going to defend against McCain... "I voted for the war but it's because I was fooled into trusting GW Bush. Now I'm for ending the war. Sort of. Some time.. Not now, but kind of soon maybe." McCain will clobber her on that one.

Obama assumes the audience is intelligent enough to not fall for those political tricks. He responds in a way that says, "Take a look at it yourself." He spoke out against the war in 2002 even though it was so unpopular to do so. That is the kind of principled leadership we need. Not only for the country in general, but to fight and defeat McCain and the republican fear-machine in general.





You are referring, I believe, to the Rasmussen Report of several weeks ago, as you are perpetrating one of the misconceptions of that report. The Rasmussen Report indicated that Clinton led McCain 47% to 45%. Obama also led McCain, and by a wider point spread, but Clinton won a larger share of the national vote in its entirety. What the poll really indicated was that either Clinton or Obama could beat McCain.

More importantly, that poll is several weeks old -- it was conducted immediately following the onslaught of negative Clinton "racist" PR, and before articles reporting on matters such as Obama's "spin" of the nonexistant legislation that supposedly required Exelon and other nuclear plants to report radioactive leaks. So, it is a little misleading to use that report to assume that Obama would garner more popular support in a general election than Obama.

Quite frankly, I believe the opposite is true. Obama is relying heavily on the independent vote to rally behind him, and it probably will. However, HRC retains the support of most core Democrats, who are (I believe?) far more likely than independents to turn out in the general election.

Nor do I believe that the Rovian Republican PR war machine is naive enough to make such a fuss about wanting, begging, pleading with the dems to run HRC if that is indeed what they truly want. The last thing the GOP wants is an election showdown with HRC. Trust.




You say that Obama is "relying on the independent vote to rally behind him." while HRC "retains the support of most core Democrats".

Forgive me, I don't follow. So you're suggesting that if Obama gets the Dem nomination, some "core Democrats" are going to decide to stay home or vote for McCain instead?

No, come the General Election, Clinton supporters (the Dem base) are going to still support Obama.

So the dem base is a non-factor in the general election.

Because, and this is the point: Obama will get them, as would Clinton, but he'll also get a lot more Ind's and some Republicans (who would otherwise vote for McCain or not at all), as well as more voters in general who Obama inspires to come out and vote (instead of staying home), as well as others who would vote Green or some other party if McCain and Clinton are the nominees.

It's simple math. In the general election the vote will be:

Clinton Vote (C) = Dem core.

Obama Vote (O) = Dem core + Independents + a few Republicans + other would-be McCain voters + would-be stay at homers + would-be Greens + would-be Nader voters, etc..

(would-be's meaning would-be if Clinton is the nominee instead of Obama)

Final math problem result:

O = > C

In English: Obama [vote] is greater than Clinton [vote].

: )

Anyway, today (yesterday now) sure was fun. Still waiting for final CA results.. Obama just now pulled within 10 points with 17% still to come in! So looks like he'll end up "losing" CA by only 9 or maybe 8 points, which is a lot closer to a tie in delegates, and which should make the final total delegate results for today almost an exact tie (not incl. Superdelegates), as I predicted.

Going to be fun from here on out!



Post a comment

Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.