February 3, 2008 - February 9, 2008


POLL: Gallup and Newsweek National, SurveyUSA Statewides

Gallup Daily Tracking

Clinton 48, Obama 43... McCain 53, Huckabee 21

(story, results; press release)

Obama 42, Clinton 41... McCain 51, Huckabee 32


Obama 52, Clinton 33... McCain 56, Huckabee 17

Washington State
Obama 50, Clinton 45... McCain 54, Huckabee 25

POLL: SurveyUSA Virginia


Obama 59, Clinton 39... McCain 45, (Romney 22), Huck 20

Without Romney:
McCain 57, Huckabee 25, Paul 9

We encourage our readers to click through to see field dates, sample sizes, margins of sampling error, target populations and addition results.

A Response from Gallup's Frank Newport

Topics: 2008 , Barack Obama , Frank Newport , Gallup , Hillary Clinton , Likely Voters , Mark Lindeman , Pollsters

In response to the dialogue we've been having about the Gallup Daily tracking survey (here and here), Gallup's editor-in-chief Frank Newport sent the following response. Say what you will about Gallup, they are consistently among the most transparent and responsive of the public pollsters.

We are always glad to discuss and analyze Gallup poll data. We generally learn from the insights, comments and questions of others.

The particular reader to whom Mark spends time responding was focusing on the fact that Gallup's daily election tracking was not in exact sync with the vote totals across the 22 Super Tuesday states.

We never reported the Daily Tracking results as projective of what would happen on Super Tuesday. Had that been our intention, we would have used a strict likely voter screen. We would have made specific assumptions about what turnout would be in each state and adjusted each state accordingly. This is what we normally do when trying to predict the actual vote in a state or national election. We did not design the tracking survey methods for that purpose. The general patterns of trends among the broad sample of voters we look are extremely important. But the exact numbers are not projections of the vote in any state or combination of states.

As we reported, candidate support levels in the Super Tuesday states were not dramatically different from the national support levels. This suggests that the momentum and trends observed nationally could be hypothesized to be reflected in the Super Tuesday states.

But for a reader to take that as a prediction by Gallup about the precise vote outcome in all Super Tuesday states (or certainly any individual state) is incorrect.

Our data suggested that among all voters across the country and in Super Tuesday states prior to Feb. 5th, Hillary Clinton had a lead over Barack Obama. Of course not all voters went to the polls -- they never do. Initial estimate are that there was only an average 30% turnout - and a turnout which varied widely across states.

The Gallup Daily election tracking uses a mild screen that filters out just those respondents who say they are not likely to vote in response to a four part question. For Republican voters in February so far that has been 16.9%. For Democratic voters it has been 13.7%. In other words, the screen leaves in more than 80% of national adults, making it functionally similar to the typical registered voter screen.

It certainly wouldn't be expected that a large sample of 80% + of all adults would mirror the actual vote total in a widely disparate group of states with on average just about 30% turnout - and with different turnout within each state. By way of example, when we retrospectively go back and look at the sample of voters from Super Tuesday States from the last five days before Super Tuesday -- screened only among those who are extremely likely to vote -- we find that the vote totals are near a tie, with Obama at 48% and Clinton at 45%.

But we didn't get into that before Super Tuesday because that was not our purpose. The purpose of the national tracking is to monitor the mood of all Democratic and all Republican voters across the country as this primary season progresses. After Jan 3rd, of course, some of these people had already voted, and that proportion continues to go up.

One of the great values of Gallup's tracking is the ability to monitor on a daily basis the changing dynamics of the campaign and to see where the momentum is. (The second value is to be able to aggregate data and look at detailed subgroup analysis). Obama had been gaining in the week or two prior to Super Tuesday to the point where he was essentially tied with Clinton among the broad sample of all voters. But then Clinton retook the momentum. Thus, we hypothesize that had the election been held on Saturday, for example, it looks like Obama would have done better than he eventually ended up doing. But we were not attempting to say what the exact vote totals would be.

[UPDATE (2/10)]: The comments left for this this entry are unusually well-expressed and definitely worth a read. They have inspired a few additional thoughts of my own (delayed, admittedly, by a much needed 36 hour break):

First, we ought not pick just on Gallup. Gallup's broad approach to selecting the "voters" that get asked presidential primary questions is more or less what the other national polls do. I first wrote about this issue almost a year ago and warned about it just last week, on the eve of Super Tuesday when headlines told us of a "dramatic shift" toward Obama.

Second, I am certainly sympathetic to the nearly insurmountable challenges that would be involved in creating a combination actual (past) voter/"likely voter"/"likely caucus goer" model that would apply at the national level and somehow take into account the myriad of different rules for participation and historically varying turnout rates. It would not be at all easy.

Also, be careful what you wish for: Those who remember Gallup's daily during the 2000 election will recall that they applied their "likely voter model" to data as early as Labor Day. Critics made a strong case that while the model works well a week before the election it introduces a lot of variation in the kinds of voters selected as "likely," much of it questionable.

Third, I agree with Mark Lindeman that there is value to Gallup's approach. "it's very interesting," he wrote, "to know what Democrats and Republicans (including leaners) around the country are thinking of "their" candidates, whether their states have already voted or not." However, I tend to agree even more with reader DTM's reaction:

[Quoting Newport] "One of the great values of Gallup's tracking is the ability to monitor on a daily basis the changing dynamics of the campaign and to see where the momentum is."

I think it is fair to say the campaigns are directed at eventually getting actual votes in caucuses and primaries, and the kind of momentum the campaigns care about is the kind of momentum that would further such an end. But given the way in which Gallup is defining "voters", the relationship between what is going on in their tracking polls and what the campaigns are actually trying to accomplish is less than clear.

And this is precisely the sort of confusion which worries me. Indeed, they seem to be more or less encouraging people to use these tracking polls for "horse race" coverage, while at the same time admitting they are not really even trying to screen for actual voters in the upcoming contests, which is what the "race" is all about.

Most people who follow the national poll numbers -- including journalists and political professionals -- treat them as if they measure the views of actual voters in party primaries or caucuses. Pollsters could do a much better job making it clear that they also include far more "leaned partisans" than are likely to actually participate in the party primaries and caucuses (regardless of what respondents claim on vote likelihood questions).

POLL: Gallup Daily Tracking

Gallup Poll

Clinton 49, Obama 42

Consumer Confidence
Economy getting worse 77, getting better 14

POLL: InsiderAdvantage Virginia

Obama 52, Clinton 37

POLL: ARG Wisconsin

American Research Group
Clinton 50, Obama 41... McCain 51, (Romney 29), Paul 7

The Limits Of Daily Numbers

Topics: 2008

My National Journal column, which discusses the Gallup Daily national survey, is now online.

POLL: TIME/SRBI National Survey

TIME/SRBI (analysis, results)

DEMS: Clinton 48, Obama 42
GE: Obama 48, McCain 41... Clinton 46, McCain 46

Re: Gallup Daily Vs. Super Tuesday

Topics: 2008 , Likely Voters

While I was finishing my National Journal column late yesterday afternoon, Gallup posted a longer than usual Gallup Daily update that answers most of the questions we asked here yesterday. It is a must read for those closely following the Gallup Daily numbers and other national surveys. Our readers blogged the key passages in the comments last night, but for those who missed it here are the key paragraphs:

The vote opinions of those in Gallup Daily tracking will not, of course, represent the actual vote in various states or in particular combinations of states on Election Day. One reason is that the tracking represents a broad sample of all respondents who say they are at least somewhat likely to vote, removing a small percentage who are unable to vote or not engaged in the campaign to any degree. The "not likely to vote" group is less than 20% in general (among both Republicans and Democrats), meaning that over 80% of American adults are included in the voter figures Gallup reports, making it similar to a typical "registered voter" figure.

Those who track voter turnout in various states that voted on Super Tuesday estimate that actual turnout was around 30%, and varied considerably among states. Thus, a broad sample of over 80% of American adults would not be expected to match the actual voting patterns of the much smaller group that turn out to vote in either party's primary.

There is, in fact, strong evidence in the tracking data from the days prior to Super Tuesday that Obama did significantly better when those who reported the highest likelihood of voting are isolated in the sample. Retrospectively, Gallup analysis can isolate just voters who say they are extremely likely to vote -- about 50% of the sample (this still overestimates actual turnout). The vote preferences of Democrats within that smaller slice for the five days prior to Super Tuesday (and after John Edwards left the race) show that Clinton (45%) and Obama (48%) were basically tied [emphasis added].

This finding is significant since it says something important, not just about the Gallup Daily tracking but about most of the other national surveys that ask about the Democratic primary vote preference among similarly broad samples (that overrepresent primary turnout). Back in April of last year, Open Left blogger Chris Bowers (then with MyDD) wondered whether these overly broad samples in national polls might be inflating Hillary Clinton's advantage. At the time narrower slices of national surveys -- like the one that Gallup did above -- did not support the theory. However, this new evidence, coupled with Obama's consistently better performance in lower-turnout caucuses on Tuesday, suggests that other national surveys may be overstating Clinton's advantage.

Two weeks ago I wondered again if the national screens are "tight" enough. This new evidence from Gallup suggests that if we are interested in the preferences and opinion of Democratic primary voters nationwide, they are not tight enough.

Reacting to these new findings, FlyOnTheWall, the Pollster reader whose question started this discussion, asked:

If Gallup is saying that the sample which includes 80% was wildly off the mark as a predictor of actual voting, but that the sample which included just the 50% of highly likely voters came darn close to predicting how actual voters actually vote - then why the heck don't they use the tighter screen all the time?

If they're trying to find out how all Americans feel, they shouldn't use any screen. But if they're tracking voter sentiment, then they should be screening for voters. And since a loose screen produces results that aren't predictive, and a tight screen produces those that are, I really wish they'd just use the tight screen going forward.

To report daily results based on a rolling average of "extremely likely" to vote respondents, Gallup would either need to call twice as many Americans every night or report a rolling six-day average in order to keep the sample size the same. Read my National Journal column later today (I will add a link when it's up) to get a sense for why it would be a bad idea to do daily tracking based on a smaller sample.

However, Fly makes a very good point. It would certainly be helpful if Gallup could report a weekly average based on just the "extremely likely" to vote respondents. Since they interview 7,000 adults a week, they are uniquely positioned to regularly compare high turnout Democrats to all the rest.

Gallup Daily vs. Super Tuesday

Topics: 2008 , Barack Obama , Gallup , Hillary Clinton , Likely Voters , National Journal

In response to a reader's question about the Gallup Daily survey, I left a comment last night that was not correct. It concerned the screen that Gallup applies to the results on the Democratic and Republican presidential primary contest. I had assumed, wrongly as it turns out, that Gallup reported the results for all adults nationwide that identify or lean Democratic. An alert reader caught and alerted me to the word "voter" used in their methodological blurb. I emailed Gallup's Jeff Jones to check, and he kindly replied with the precise explanation for how the select the primary "voters" whose preferences they report every day:

Republicans or Republican-leaning independents who say they are extremely, very or somewhat likely to vote in their state’s primary or caucus when it is held.

Democrats or Democratic-leaning independents who say they are extremely, very or somewhat likely to vote in their state’s primary or caucus when it is held.

We [also] make provisions for those residing in states that have already held their primary caucus – those who indicate they have already voted are considered extremely likely to vote, and those who did not vote in their state’s primary or caucus would be excluded from the base.

One important note: The screen that Jones describes is similar to what other pollsters use in statewide surveys, but it is not the more rigorous and sometimes controversial Gallup "likely voter model" that they use in general elections and used for their surveys in New Hampshire.

Back to the question from "FlyOnTheWall" that prompted this discussion:

Today's Gallup polling was done yesterday [Tuesday]. It's of likely Democratic primary voters. And it attempts to show for whom they're going to vote. Today's snapshot shows a 13-point lead for Clinton.

Only we ran this experiment on a broader basis yesterday, and found (based on the tallies of the popular vote that I've seen) less than a point separating the two candidates. And it gets worse. Gallup broke out the February 5 states a few days ago, and found that voters there were more - not less - favorably disposed to Clinton than their entire sample. So, presumably, what Gallup is telling us is that voters in February 5 states favor Clinton by some 15 points - when the voters themselves turn out to be evenly divided.

That's as an egregious an error as Zogby, from a pollster who's supposed to be a whole lot more reputable. What gives?

And that's a fair question. First, to clarify, the results released yesterday that showed the 13-point lead were based on interviews conducted from Sunday afternoon (before the Super Bowl started) through Tuesday night. While some Tuesday night respondents on the West Coast may have been aware of the results, most were not. So Fly is right to suggest that the Sunday to Tuesday window was a good time period to compare to the actual results.

Of course, Gallup did not report on the vote preference of voters in Super Tuesday states in their Sunday to Tuesday data. They did that on Monday:

Forty-nine percent of Democrats and (where eligible to participate) Democratic-leaning independents in Super Tuesday states favor Clinton for the nomination, while 44% choose Obama. This analysis is based on tracking data from Jan. 30-Feb. 3, all collected since John Edwards suspended his campaign.

But note the last sentence. They reported a result nationally on Monday from the last three-nights of interviewing (Friday to Sunday) showing Clinton 4 points ahead of Obama (47% to 43%). However, the results for the Super Tuesday states were culled from interviews over the prior five nights of calling (Wednesday through Sunday). The different time period might have made a difference, although the national Clinton-Obama margin looks to have been roughly four points over the five day period as well. Perhaps Gallup can clarify.

In the spirit of my op-ed piece this morning, the disclosure of some additional statistics from the Gallup data would help in comparing the Super Tuesday results to the Gallup Daily results of the last week or so:

  • What percentage of adults, nationally, qualified as "Democratic and Democratic-leaning voters" over the last week or so? What percentage of adults qualified as "Republican and Republican-leaning voters?" How does the combination of the two compare to the 29.1% turnout of eligible adults that Michael McDonald's invaluable primary turnout web page estimates for Super Tuesday?
  • Gallup may have to reach back more than a week to get sufficient sample sizes, but what are the same statistics when we compare Super Tuesday primary states to Super Tuesday caucus states? McDonald's data indicates the obvious -- that caucus turnout was significantly lower than primary turnout -- though I will need to crunch the data more to get an overall comparison of turnout in Democratic primaries to Democratic caucuses.
  • And while we are at it, what was the vote preference over the last five or six nights of interviewing if we looked only at primary states that voted on Super Tuesday: And how does that preference compare to the actual votes cast in just the primary states?
  • [Update - I left one out: Did "extremely likely" Democrats in Super Tuesday states differ from those just somewhat likely to vote?]

Perhaps someone at Gallup could take a crack at this. It would make a terrific Gallup Guru item, don't you think?

A New York Times Op-Ed...and an Eplogue

Topics: 2008 , Disclosure , John Zogby , NY Times , SurveyUSA

I am honored to report that I have op-ed column in today's New York Times on a subject that will be familiar to regular readers: The need for better disclosure of methodological details by pollsters. I hope you will go read it all.

As it happens, John Zogby provides an epilogue on his hugely inaccurate California survey in his post Super Tuesday commentary:

About California: Some of you may have noticed our pre-election polling differed from the actual results. It appears that we underestimated Hispanic turnout and overestimated the importance of younger Hispanic voters. We also overestimated turnout among African-American voters. Those of you who have been following our work know that we have gotten 13 out of 17 races right this year, and so many others over the years. This does happen.

So now he tells us. Although, if you notice, he is still not ready to disclose the racial and ethnic composition of his California survey. By how much, exactly, did they "underestimate" HIspanic turnout and "overestimate" the contribution of younger Hispanics and African Americans? He did not make these details available on his survey release on Tuesday (at least not to non-subscribers), and is apparently not making them available now. SurveyUSA, Field, McClatchy/Mason Dixon, and Suffolk University did report demographic composition details. That ought to tell us something.

Incidentally, it is also worth noting that while the results of the final SurveyUSA poll nailed the final ten-point Clinton margin, and the "sturdy" 13-point Obama lead forecast by Zogby never materialized, in Missouri the roles were reversed. In the final hours, SurveyUSA showed Clinton leading by eleven <s>nine</s> points (54% to 43%), while Zogby gave Obama a slight advantage (45% to 42%). Obama won 49% of the actual vote to 48% for Clinton.

The lesson?: Better disclosure puts us in a better position to understand and interpret the data, but all pollsters are fallible and all polls are subject to error (random and otherwise).

POLL: Gallup Daily Tracking

Clinton 52, Obama 39... McCain 41, Romney 25, Huck 21

See Gallup's full daily tracking trends for both parties here.

Super Tuesday Live Blogging

Topics: 2008 , Barack Obama , CBS , Chuck Todd , CNN , Exit Polls , Hillary Clinton , Mark Lindeman , Mason-Dixon , MSNBC , Pollster.com , Rasmussen , Suffolk , SurveyUSA

We are live blogging tonight and posting the most recent estimates for each race derived from the official public exit poll tabulations as they are posted online. The table will update regularly (though you will need to reload the page to see updates), and live-blog updates will appear below in reverse chronological order below the table and its caveats below. All times are Eastern.

(If you do not see a chart, click here.)

stliveblogdems.png stliveblogreps.png

CAVEATS: The "exit poll estimates" here are extrapolated from publicly available tabulations; they are not "insider information" about up-to-the-minute projections. These estimates are never based upon interview data alone. Tabulations posted as the polls close incorporate pre-election expectations (based on pre-election polls); during the night, the tabulations are updated (occasionally, not continuously) to reflect vote count data. The results posted here may not always reflect the very latest public information, although we hope to keep them close at least through midnight Eastern. Keep in mind that for the first few hours after the polls close these will be survey based estimates, and surveys are subject to error (some random, some from other sources). We encourage you to visit the complete tabulations available at the following links from CBS, CNN and MSNBC.

Candidates projected as winners by the networks are in bold.

1:19 - It is late, we are exhausted and the results seem reasonably settled, so we are going to call it a night. Sometime later today, the exit pollsters will update each cross-tabulation so that the overall result will match the actual vote count. As such, we will probably not do a final update of these tables.

Please join me in thanking Mark Lindeman (who crunched the numbers and did all of the updates) and Eric Dienstfrey (who set up the table so Mark could update it). And thank you Pollster.com readers for nearly 74,952 unique visits and 372,000 page views yesterday. Goodnight!

1:00 - Just to update the earlier discussion of delegates. On MSNBC, Chuck Todd just reported delegate "estimates" for tonight that include California and New Mexico: 841 for Obama, 837 for Clinton, with what he described as a "plus or minus" of 10 on the California delegates (and as Anon notes in a comment, "that's just the [estimated] number of delegates won tonight, not a running total"). I'm assuming other news organizations are or will be reporting similar estimates; please post those to the comments if you catch them.

12:56 - Back online. One commenter is reporting problems viewing the table with the Firefox browser. We tested it in several browsers before going live and I have been viewing on Firefox all night. If anyone is having trouble, please email us with your setup info (to questions at pollster dot com).

12:08 - I will be off the grid for about 30 minutes, although Mark Lindeman will continue to post updates.

12:05 - Anon posts some numbers gleaned from past exit polls:

  • 2006 Dem gubernatorial primary - 12% Latino, 8% black (LAT)
  • 2004 Dem presidential primary - 16% Latino, 8% black (via CNN)
  • 2000 Dem presidential primary - 17% Latino, 11% black (via CNN)
  • 1998 Dem gubernatorial primary - 12% Latino, 14% black (LAT)

12:00 - Mark just posted an update of the CA tabulations. One thing to keep in mind is that the exit poll analysts need hard vote counts to determine two very important things: The appropriate mix of early and polling place votes, and the correct regional distribution of the turnout. Adding hard data will either firm up the racial composition numbers reported below, or change them. My guess is that their level of confidence in those racial composition numbers will determine when they "call" a winner in CA.

11:55 - "Pragmatic Progressive" corrects me on the Chuck Todd delegate report I posted at 11:25. Todd projected a total of 1140 delegates, but (assuming I grabbed the right numbers) there are 1,681 delegates up for grabs tonight. Todd's estimate does not yet include roughly 546 delegates from California and some of the other Western states.

11:41 - As of the first report, the California exit poll shows Latinos contributing a larger percentage of the vote, and African Americans a smaller percentage of the vote, than indicated in the pre-election polls.

Latinos are currently showing as 29% of the Democratic vote in California on the exit poll tabulations. In the final pre-elections polls they were:

African Americans are currently showing as 6% of the Democratic vote on the exit poll tabulations. In the final pre-election polls they were:

  • 12% on Field
  • 9% on SurveyUSA
  • 9% on Mason-Dixon
  • 8% on Suffolk

As elsewhere, Clinton wins the vast majority of Latinos in California (65% as of now) and Obama wins a greater share of African-Americans (78%), so if those composition numbers hold, it bodes very well for Clinton. Update: Just noticed that SN made a similar observation in the comments as I was typing up these numbers.

11:25 - Daniel T made this point about an hour ago in the comments:

"[W]inning" states in the primary means nothing since no democratic states are winner take all. The real key is the delegate count and while Clinton is still leading there, the night is still young.

He is right, of course, which is why I will pass on something that Chuck Todd just reported on MSNBC. Their estimates are that when all votes are counted, the delegates at stake tonight will split 595 for Obama and 546 for Clintons, so "we're looking at an even split" in the total delegate count tomorrow. [Correction: I heard that wrong. Todd's totals apparently do not include CA and other western states -- see my 11:55 update].

11:12 - A quick summary table of the current tabulations among white voters as reported on the exit polls (and these are not extrapolations but the report among whites in each state):

02-05 exits by race.png

11:05 - Mark L just added the CA estimates.

10:50 - Oops. Looks like we overlooked New Mexico for the Democrats. Updating shortly. Thanks to Anon and Daniel for the edit.

10:40 - Mark L just posted more recent updates from most of the states. The additional actual vote counts have changed some of the estimates considerably. For example, Clinton's margin in Massachusetts the margin is now roughly 15 points.

9:32 - Another apparently slow-to-update estimate is New Jersey. MSNBC has called the state for Clinton, but the update has not yet updated since we last checked at 8:30 and found Clinton receiving less than a percentage point more than Obama.

9:12 - For what it's worth, regarding the spin and counter-spin now underway over Massachusetts: We reported only a handful of pre-election polls there in recent weeks. The Suffolk University/ News Poll had the two candidates essentially deadlocked (Obama 46%, Clinton 44%), Rasmussen Reports (a week ago) had Clinton leading by six points (43% to 37%) and Sur veyUSA had Clinton leading by 17 percentage points (56% to 39%). The current estimate is Clinton by 5 (50% to 40%). Of course, that estimate will likely change as more hard count comes in. [UPDATE - Not so fast...the 10:36 update increases the Clinton lead to roughly 15 points (56% to 41%)].

9:05 - Josh asks:

MSNBC just called MA for Clinton... any idea what they believe that you don't at this stage?

What Josh noticed (that MSNBC called Massachusetts while the estimate we had here still showed Obama slightly ahead) underscores the point I made at 7:30. The exit pollsters update the cross-tabulations less frequently than the estimates they use to make projections. They updated the MA cross-tabulations online just as the call was going on the air. And the updated estimate in Massachusetts -- which is now undoubtedly based on a lot of actual vote count in the sampled precincts -- has Clinton ahead by roughly five points.

8:56 - Two things: First, as should be obvious, we are not bothering with information that you can get faster and better from the networks. Second, I have been watching MSNBC, and just heard our friend Chuck Todd reading projected delegate splits from various states. Needless to say, those counts are the most important numbers tonight.

8:51 - Just saw this comment posted earlier by our colleague Marc Ambinder about those leaked early exit polls (that appear, not here, but on sites like the one who's name rhymes with "budge"):

Fellow journalists and pundits. I have the same data you have... and I would just remind all of you that the first wave of exit poll data is not reportable or reported for a simple reason: the sample sizes are not large enough to accurately tell us much of anything, unless one candidate is getting, like, 80% of the vote.

That's about right.

8:45 - One note: An informed source advised earlier today that the polling place "exit" interviews will be supplemented tonight with telephone surveys conducted among early and absentee voters by the network consortium in California, Arizona and Tennessee.

8:42 - In the comments, Daniel T suggests:

it would be helpful if you but in italics or bold the number of the "projected" winner for that state

Great suggestion - consider it done (literally). We have highlighted candidates projected as winners by the networks (or at least one of them) in bold.

Josh asks:

Would it be possible for you to highlight the cells in the spreadsheet according to who is leading in each state? It would make it easier to follow as the estimates shift throughout the

Possible, obviously, but we would rather not, as we're really not sure that someone "leading" is really leading (we do not have access to the confidence level assigned to each margin). Daniel T also asks:

Could you add a column that shows the number of delegates atstake in each state. I think there is room. It would be helpful.

Sigh. Yes, it would, but that one will have to wait for Super Tuesday 2009 (or perhaps next week). The table ain't broke, so we are not going to try to fix it. But thanks for all of the suggestions.

8:27 - So I've alluded to the fact that these estimates improve over the course of the evening. What does that mean? It means that as the polls close, the estimates are based on some combination of results from the exit poll interviews and (believe it or not) pre-election poll averages. Once the polls close, the interviewers attempt to obtain actual results for their sampled precinct (or another "reporter" attempts to get the results from the county or state registrar). The exit poll analysts use these numbers to do two things: First, they gradually replace the exit poll results in their estimate models with the actual count precinct-by-precinct. They also calculate the "within precinct error" statistic for each state (or regions with each state - not sure) that are used to adjust exit poll results from the other precincts where actual count is not yet available.

The exit pollsters also have "reporters" who gather hard vote counts from a much larger random sample of precincts. All of this data goes into the computer models and is used to create various estimates of the vote. The "decision desk" analysts look at all of those estimates in deciding whether to "call" a race.

A separate operation within exit-poll-central takes whatever estimate they deem most trustworthy and uses it to weight the subgroup tabulations that we can read online. And we take those tabulations as they appear online extrapolate the estimates above from those tabulations. They typically do one updates 30 to 60 minutes after the polls close and another two or three over the course of the evening. All of this is a long-winded way of saying that what you are seeing here is not as current as the information the network analysts are using to make their calls.

8:13 - Mark L has just posted more estimates, and it is worth stressing something important: Most of these margins of very close, well within the "margin of sampling error." Needless to say, the networks are not yet ready to project and will make their calls as hard returns become available. A difference of a point or two is really meaningless at this point.

8:00 - Polls just closed in Alabama, Connecticut, Delaware, Illinois, Massachusetts, Missouri, New Jersey, Oklahoma and Tennessee. We should have updates for these soon.

7:51 - "Anon" comments:

you'll get slightly different numbers depending on which breakdowns (age,
race, geography, etc) you plug into the equation. your numbers are prettyclose to mark's, though i don't know which one(s) he's using.

Thanks for asking. You remind of two things I neglected to mention. First, Anon knows that the brains behind these tabulations is Mark Lindeman, an assistant professor of political studies at Bard College in Annandale-on-Hudson. Mark is the one doing the extrapolating. The table that runs with a Google Docs spreadsheet, was put up by the indefatigable Eric Dienstfrey. Thank you Mark and Eric!

Second, Mark has put together a program that does not one, but more than 20 extrapolations and then calculates the average of all of them. The point is that the extrapolation introduces rounding error, so averaging a bunch of calculations minimizes that potential.

7:44 - So what's the point of posting the estimates extrapolated from the exit poll tabulations? One reason is that those estimates, imperfect as they are, provide what is arguably the best projections available of the final result. Again, and we cannot stress this enough, those estimates are subject to change and should improve over the course of the night. But in watching the extrapolated estimates after the polls closed since in all of the primaries so far this year, I can say that they provide a far more accurate data on the outcome than the first waves of actual vote that crawl across the bottom of the network broadcasts.

7:38 - The Associated Press has its first review of the national exit poll data up (via The Page).

7:31- A certain web site (rhymes with "Grudge") has a bunch of leaked exit poll estimates in states whose polls have not yet closed. I'd recommend to those curious about those leaked numbers -- and everyone else, for that matter -- to review my post from earlier today. Remember, it's just a survey, the networks rarely "call" an election on the exit poll alone unless the leader's margin is *very* large (i.e. a "lead" of 3-4 points isn't enough to assure a victory) and the early estimates (as well as the tabulations we're extrapolating from above) are usually weighted by something called the "composite estimate" which includes an average of pre-election polls.

One more thing to keep in mind: The leaked numbers from the West coast states are extremely preliminary. Remember, it's only 3:30 in the afternoon there. These are mostly from the first wave of reported exit poll data (there are three). Experience has shown that the first two waves can be unreliable, but the numbers usually settle down in the third wave that reports just before the polls close.

Somebody's Gonna Be Wrong

Topics: 2008 , Divergent Polls

We certainly have a deluge of recent polls to consider from the states now voting (or that will caucus later tonight, but the highly divergent results on the final round of poll on the Democratic race in California caught my eye first today (as well as many readers who have left comments). I want to start there and, since time is too short this afternoon to delve into the minutia of the California polls, make a more general point about disclosure the bottom line of all this variation.

02-05 final CA Dems.png

The final California poll from SurveyUSA (and five local television stations in California), conducted Sunday and Monday, has Hillary Clinton leading Barack Obama by 10 percentage points (52% to 42%), while the final poll from Zogby/Reuters/C-Span shows Obama leading Clinton by 13 points (49% to 36%). Four other surveys released over the weekend show results somewhere in between. [Elsewhere on Pollster, Charles Franklin charts compares the differences among pollster in California for both the Republican and Democratic candidates].

Needless to say, random sampling error alone (the so-called "margin of error") does not explain these discrepancies. They result from some difference in the methodologies of the two polls -- who they interviewed, how they sampled and interviewed them, the questions they asked, how the selected "likely voters" and how they weighted the results. If we knew more about how these polls differed, we might not be able to predict a winner, but we would at least have a better understanding of the factors that will determine the outcome of the race.

The analysts at the Field poll, in their report on their final survey (conducted over seven days, from 1/25 to 2/1) had a helpful explanation of the variables that will affect the outcome of the election:

[Consider] that 12% of the Democratic voters and 6% of the Republicans either already voted for or expressed a preference for a candidate other than those who remain in the running. According to the poll, over half of these voters were early mail ballot voters, and have already sent them in. Yet, for those voters who haven’t yet cast their ballot, there is some uncertainty as to whether they will follow through and support a candidate who has withdrawn from the race. They may not vote at all or switch to another candidate.

In both major party races, there are also unusually large proportions of voters – 18% in the Democratic primary and 15% in the Republican – who were undecided in the final days of the campaign. These voters had indicated in prior questions that they were highly likely to vote. Considering the many months of campaigning, extensive free media coverage, advertising, and recent televised debates, these voters have been exposed to considerable information about the candidates and appear to be in some real conflict as to whom to support. How these voters come to judgment will have a big bearing on the election outcome of both sides.

There is another aspect to the Democratic primary findings that is unique and where there is not much precedence in previous presidential elections. It is the group of non-partisans who say they will vote in the Democratic primary. These voters have candidate preferences that counter those of registered Democrats. The relative size of each eventual voting bloc, therefore, will have a major impact on the outcome.

Further, there are unusually large differences in candidate preferences among some of the standard voter sub-groups of each party. To pose just one example from each of the Democratic and Republican contests: (1) the large divisions in support between men and women voters in the Democratic primary, and (2) the big split in preferences between strong conservatives and Republicans who are not strong conservatives. In each of these examples, it would not take large changes in voter turnout proportions from those shown in this report to produce a very different election outcome.

All of these factors mean that methodologies matter: The extent to which an interviewer or interviewing method "pushes" voters to make a decision and the mix of demographic groups the pollsters sample will both directly affect where the candidates stand.

I started to write this entry to focus on the two most divergent California polls, but then realized (with the help of a alert reader) that polls in other states are showing similar variation. For example,

  • In California, the margins reported by various pollsters for the Democrats over the last week vary by 23 points.
  • In Alabama, for the Republicans, the margins (between McCain and Huckabee) vary by 24 points
  • In Illinois, for the Democrats, the margins vary by 25 points
  • In Massachusetts, the margins for the Democrats vary by 19 points
  • In Connecticut, the margins for the Democrats vary by 15 points

I could walk through what we know about the methodologies of the SurveyUSA and Zogby polls in California, but it would not resolve the temporary mystery of which will be judged most accurate when the votes are counted. The reason, to focus for a moment on just these two polls, is that while SurveyUSA discloses a wealth of information in their interactive crosstabs and their online report -- how they sampled, what rough percentage of adults their sample represents, what the composition of their survey is in terms of gender, age, race and region -- we know next to nothing comparable about the Zogby poll. And unfortunately, while the Field Poll in California also produces a highly informative report, and while McClatchy released a filled-in questionnaire from Mason-Dixon that includes demographic composition data, the Zogby poll is more indicative of the general lack of disclosure elsewhere.

If ever there was a case for better methdological disclosure by pollsters, this is it. If one poll, conducted entirely over the last 48 hours, shows a candidate leading by 10 points, while another conducted in the same state over the same time period, shows a another candidate leading by 13, and we cannot see enough of the details of how the polls were done to at least explain why they differ, why should we trust what any of these polls tells us?

Typo corrected (thank you, Kevin).

California Super Tuesday Pollster Comparison

Topics: 2008


There is quite a bit of disagreement in the California polls, so let's do a Pollster Comparison to see who says what.

The Democratic race has the biggest gap across pollsters. Most see Clinton ahead, but disagree on how much the race has tightened. But Suffolk and Rasmussen put Obama just barely ahead in their final polls. And the big difference is in the Zogby/Reuters/C-SPAN poll that sees Obama not only ahead but expanding his lead to double digits.

That gives us a range of results from Obama ahead by more than 10 to Clinton ahead by 10. Somebody is not quite right here. But who? And why?

Across all the polls, Clinton is relatively flat while Obama has gained. But looking at each poll and each candidate reveals more differences across pollsters.

Both Zogby and the Field poll find some decline for Clinton while others have her flat or rising (SurveyUSA and Rasmussen).


Everyone sees Obama either flat or rising, with no pollster (who has at least 2 polls) seeng Obama declining. But the extent of the rise, and how high Obama's starting point was, differs quite a bit across pollsters. Survey USA and Rasmussen both see Obama gains, but SurveyUSA offsets that by measuring gains for Clinton as well, resulting in relatively little net change over their last three polls. Rasmussen, on the other hand, has Clinton gains but even bigger Obama gains, resulting in a net decline and a final poll with Obama just barely ahead.


The Zogby polls in contrast find both Clinton declines AND Obama gains, making for a net change that strongly favors Obama and results in his 13 point lead in the final Zogby poll for California.

On the Republican side, we also see substantial differences. Two pollsters see a dramatic tightening of the race. Rasmussen and SurveyUSA estimate that McCain's earlier lead has now collapsed to a dead heat. (A single ARG poll also sees a dead heat, but has no over time data for comparison of trend.) In contrast the LA Times/CNN/Politico and Field polls both find McCain gaining some ground in their polls. These two are a bit stale now, but showed gains even when Rasmussen and SurveyUSA saw declines, so this is not only a difference of survey dates.

Zogby/Reuters/CSPAN is again quite different, finding Romney leading in all three polls and expanding his lead to high single-digits in the last two polls.


Across the Republican polls, the differences are almost all matters of the degree of increase for both McCain and Romney. All polls agree that McCain has been increasing his support. But they also agree that Romney is rising sharply. The differences in net support have a lot to do with differing estimates of how fast and how far Romney has risen. Those with the greatest Romney rises have him catching McCain or leading him.

Given the length of time it is likely to take to count the California vote, it may be Thursday before we know which of these polls was closer to the end result.



Cross-posted at Political Arithmetik.

Looking for Super Tuesday Exit Polls?

Topics: 2008 , Exit Polls

Looking for leaked exit poll results from the Super Tuesday states? As regular readers know, the leaks of exit poll we experienced in 2004 and prior years have been stopped, as the the network consortium that conducts the exit polls now restricts access to a small number of analysts in a "quarantine room" for most of the day and did not release the results to the networks and subscriber news organizations until 5:00 p.m. eastern time. While some of that information will leak a leak, that process will remain in place today (Tom Webster, an employee of Edison Research, has blogged some details about life inside the quarantine room).

What this means (and those of you who play the various election "futures" or predictions markets should listen closely) is that any "exit poll" numbers you hear about before 5:00 p.m. are bogus (or at least, not produced by the networks).

Here are a few tips for making sense of the exit poll data you will see tonight (a slightly edited version of tips I posted on the morning of the New Hampshire primary, with a few edits):

1) An exit poll is just a survey. Like other surveys, it is subject to random sampling error and, as those who follow exit polls now understand, occasional problems with non-response bias. In New Hampshire (in 1992) and Arizona (in 1996)* primary election exit polls overstated support for Patrick Buchanan, probably because his more enthusiastic supporters were more willing to be interviewed (and for those tempted to hit he comment button, yes, I know that some believe those past errors suggest massive vote fraud -- I have written about that subject at great length).

2) The networks rarely "call" an election on exit poll results alone. The decision desk analysts require a very high degree of statistical confidence (at least 99.5%) before they will consider calling a winner (the ordinary "margin of error" on pre-election polls typically uses a 95% confidence level). They will also wait for actual results if the exit poll is very different from pre-election poll trends. So a single-digit margin on an exit poll is almost never sufficient to say that a particular candidate will win.

3) Watch out for "The Composite." As they have for the earlier primaries, we expect the web sites of CNN, MSNBC and CBS to post exit poll tabulations shortly after the polls close that will update as the election night wears on (we will post links and commentary here, so we hope you'll plan to check back in later tonight). Those data are weighted to whatever estimate of the outcome the analysts have greatest confidence in at any moment. By the end of the night, the tabulations will be weighted to the official count. Typically, the first waves of exit poll tabulations (including most that leak before the polls close) are weighted to something called the "Composite Estimate," a combination of the exit poll data alone and a "Prior Estimate" that is based largely on pre-election poll results. So if you look to extrapolate from the initial tabulations posted on MSNBC or CNN (as we have done here at Pollster each primary night this year), just keep in mind that in the estimate of each candidate's standing in the initial reports will likely mix exit poll and the pre-election poll estimates (not unlike the kind we report here).

Finally, if you would like more information on how exit polls are conducted, you may want to revisit a Mystery Pollster classic: Exit Polls - What You Should Know. Happy Super Tuesday!

Update: We're live blogging tonight with overall candidate estimates derived from the public exit polls.

POLL: SurveyUSA, Zogby, InsiderAdvantage


Clinton 52, Obama 42... McCain 39, Romney 38


New Polls in California, New York, New Jersey, Georgia, and Missouri.


New Polls in Georgia and Alabama.

Pre-Super Tuesday "Outliers"

Topics: 2008

The American Association for Public Opinion Research (AAPOR) named the members of its previously announced panel to study the New Hampshire and other pre-election polls in 2008. The panel includes our own Charles Franklin.

Frank Newport blogs six more things you may not have noticed about the 2008 election.

Kathy Frankovic says Democrats are ready to rally around a nominee.

Tom Webster shares the inside story of the exit poll "quarantine room."

Peter Steinfels looks at the controversy over questions about religious affiliation found on exit polls of Republicans but not Democrats.

Obama pollster Joel Benenson gets a New York Times profile.

Karl Rove thinks "bad exit polls shape coverage."

Brian Schaffner is using poll numbers gleaned from Pollster.com to project Super Tuesday delegate allocations for Democrats and Republicans (via Reader Maryann).

The Market Research Association (MRA) has a release out decrying so-called "push polls."

POLL: SurveyUSA, InsiderAdvantage, Capital Survey Research Center


New Polls in California, Massachusetts, Alabama, Missouri, Illinois, and Oklahoma

Insider Advantage

New Polls in Georgia and Alabama

Capital Survey Research Center

New Poll in Alabama

Full data available on our chart pages.


Public Policy Polling (D)

Obama 53, Clinton 37... McCain 31, Romney 29, Huck 27

Clinton 56, Obama 34... McCain 34, Huck 28, Romney 22

New York State
Clinton 51, Obama 32... McCain 49, Romney 24

Gallup Daily on the 2/5 States

Topics: 2008 , Barack Obama , Gallup , Hillary Clinton , John Edwards , The 2008 Race

Today's Gallup Daily release is up and they drill down to answer a question I posed earlier in my "Four Cautions" post:

An analysis of the preferences of eligible Democratic primary voters in the 23 states holding Democratic primaries or caucuses Tuesday finds results similar to the overall national results. Forty-nine percent of Democrats and (where eligible to participate) Democratic-leaning independents in Super Tuesday states favor Clinton for the nomination, while 44% choose Obama. This analysis is based on tracking data from Jan. 30-Feb. 3, all collected since John Edwards suspended his campaign.

The chart below shows their overall results.

02-04a_Gallup Daily_ Tracking Election 2008-3.png

As long as we're asking, here are a few more questions I wish our friends at Gallup would answer with the voluminous data set they have collected.

Obama's support has increased to roughly 42% over the last six nights of the Gallup survey from 33% over the six nights before that. Gallup has roughly approximately 2,400 interviews among Democrats and Democratic leaners for each six night period. So as long as we are posing questions, here are a few more I wish Gallup would try to use that data to answer:

  • Where, demographically, have Obama's gains come from? They looked at gender on Friday. What about race? age? education? income? Has Obama gaining more among hard core Democrats or just independents who lean Democratic?
  • Does the Obama-Clinton race look any different over the last six nights if they narrow the sample based on self-reported intent to vote or interest in the campaign? Obviously, that sort of analysis requires questions about vote intent or interest that they may not be asking, but they must be asking some measure of intent on the Gallup Daily.
  • What about the six caucus states? It might require rolling together more than a weeks' worth of data, but how do Obama-Clinton preferences in those states compare to the preferences elsewhere?



Obama 48, Clinton 46... McCain 52, Romney 30

New Jersey
Clinton 52, Obama 41... McCain 54, Romney 25

New York
Clinton 56, Obama 38... McCain 56, Romney 23

Obama 66, Clinton 30

Re: One Unanswered Question

Topics: 2008 , Al Gore , Barack Obama , CBS , CBS/New York Times , Fox News , George Bush , Hillary Clinton , Kathy Frankovic , LA Times , National Journal , The 2008 Race

An update on my National Journal column last week, in which I wrote:

Obama's central challenge, however, is neither the debate about which candidate can best change Washington nor Clinton's perceived advantage in experience. Rather, the most important question is whether Obama, as that same CBS News/New York Times survey puts it, "has prepared himself well enough for the job of President and all the issues a President has to face," or whether "he needs a few more years to prepare?" A few days after the New Hampshire primary, a majority of Democratic primary voters (53 percent) believed Obama still needed more time; 40 percent said he was ready.

The new CBS News survey released yesterday updates that result, and they show a statistically significant, ten-point increase (from 40% to 50%) in the percentage who say that Obama is "prepared," while the percentage who say he needs more preparation declined from 53% to 46%.

Back in December, CBS News director of surveys Kathy Frankovic provided some additional context for this question:

In September 2000, the Los Angeles Times asked likely voters to say which candidate -- George W. Bush or Al Gore -- “has the best experience for the job.” Sixty-two percent chose Gore, and only 25 percent chose Bush. Likely voters in an October Fox News Poll also chose Gore over Bush, 54 percent to 31 percent, on having the “right kind of experience.”

But while most voters thought Gore had more of the “right” or the “best” experience, a majority had already decided that Bush met the threshold. CBS News had asked registered voters in March of that year whether Bush and Gore each had “the right kind of experience to be a good president.” And for most voters, both of them did. Slightly more (70 percent) said Gore did than said Bush did (62 percent), but Bush had clearly met the experience threshold with nearly two-thirds of voters.

Two Ways of Looking At Hirshman's 16 Ways

Topics: 2008 , Hillary Clinton , John Edwards , The 2008 Race

It's worth checking out Linda Hirshman's piece in the NYT Magazine this weekend, in which she makes 16 different observations about women voters. Hirshman (best known for her "Get To Work" manifesto) makes many strong points here-such as the overemphasizing of voter gender in press coverage of the Democratic primary, and the underemphasizing of socioeconomic and other demographic differences. Indeed, these are topics I've also covered here and here. Her takes on gender differences in news consumption and political knowledge, while beyond the scope of pollster.com, are also worth reading.

But two of the 16 "ways" also relate to some of our work here, and so merit a little further discussion.

Clinton's presence is driving women's turnout?

First, "Way 11" says "Sisterhood is Power," by showing that women's participation in the Democratic primaries has increased since 2004. Last week I wrote that women's participation has been mixed when compared to 2000, but we did not look at 2004, since with an incumbent in office it was a less comparable election. Below is a table that includes the 2004 exit polls to which Hirshman refers.


Indeed, she is right-women's participation has increased since 2004. But since the 2000 percentages were as high or higher, that makes me think it is perhaps not Clinton's presence on the ballot alone that is increasing women's participation. Of course, there are other factors at work. There is record turnout this year in the Democratic primaries. Perhaps new surges of all kinds of voters (younger, African-Americans, women) render the percentage exercise not particularly useful for the Democratic primaries. At any rate, I'll include the 2004 numbers in all future discussions of this topic.

How do we measure the influence of unmarried women?

Second, "Way 15" says "The Political is Personal," and in passing uses a quote suggesting that a swing in the vote among unmarried women from Iowa to New Hampshire shows how influential this group could be. While I'm not arguing that unmarried women are not or could not be influential as a voting bloc, I don't think that particular finding is sufficient evidence. It would leave us in an awkward position of declaring unmarried women unimportant if their vote was consistent, or comparable to married women's vote. The table below shows exit poll results among married and unmarried women for each of the states for which data are available (Edwards is included since they are also states where he did well).


It seems like Iowa was an anomaly. It is the only state where the two groups of women disagreed on the winner. In every other state, Clinton's margin over Obama (or Obama's over Clinton) is not that different across that marital status/gender grouping. Given what we know about younger voters trending toward Obama, I would imagine that older, divorced women vote differently than younger, never married women. Since they are all "unmarried women" it's harder to identify exactly what is happening. If we want to argue the importance of unmarried women's contribution, it should be based on general election predictions or turnout numbers, not the primary vote (or else separate the unmarrieds by age).

But these are both fairly minor points. To me, the more ways people are looking at the women's vote, the better.

POLL: Cook/RT National, Strategic Vision (R) NJ/GA, Suffolk MA

Cook Political Report/RT Strategies

Obama 43, Clinton 37... McCain 39, Romney 24

Strategic Vision (R)

New Jersey
Clinton 47, Obama 41... McCain 55, Romney 25

Oabam 49, Clinton 27... McCain 31, Romney 29, Huck 26

Suffolk University (release)

Obama 46, Clinton 44... Romney 50, McCain 37

Re: Will Pollsters Call During the Superbowl

Topics: 2008 , ABC/Washington Post , Barack Obama , Gallup , Hillary Clinton , Pollsters , Sampling , The 2008 Race

On Friday I posted a brief item wondering about what effect last night's Super Bowl broadcast might have on number collected over the weekend. Gallup emailed to say they would not call during the game, and I assume most pollsters that were in the field last night did the same. I wondered about the demographics of game watchers as measured by an ABC/Washington Post poll last week. Late last night, the Post's Jon Cohen emailed with the following breakdown of those who planned to watch the game:

  • 72 percent of men, 50 percent of women.
  • Two-thirds of those under 45; half of seniors.
  • 76 percent of men 18-45
  • 69 percent of those with household incomes $75K+, 51 percent of thoseunder $25K

As I explained on Friday, my guess is that the inability to call for four or five hours last night probably had little effect on the poll results we are seeing now. Still the demographic profile identified above looks considerably more like Obama supporters than Clinton supporters.

POLL: CNN National, Quinnipiac NJ/NY, ARG CA, Zogby


Obama 49, Clinton 46... McCain 44, Romney 29


New Jersey
Clinton 48, Obama 43... McCain 52, Romney 30

New York
Clinton 53, Obama 39... McCain 54, Romney 22

American Research Group

Clinton 47, Obama 39... Romney 33, McCain 32

Zogby Daily Tracking

February 5 Polls: Four Cautions

Topics: 2008 , ABC/Washington Post , Barack Obama , Blogosphere , CBS , Gallup , Hillary Clinton , John McCain , Likely Voters , National Journal , The 2008 Race

Over the last 48 hours we have had an avalanche of new polls,** and given the discussion both in our comments section and elsewhere across the blogosphere, everyone seems unsure of what to make of the results and what they say about where things stand, especially in the Democratic presidential race. As is evident from our charts, the trends are highly favorable to both John McCain and Barack Obama, but from there things get murkier, especially in the Democratic race. Here is my sense of what the poll results tell us and what they do not.

The Republican race is easier to gauge, largely because of the "winner-take-all" rules that apply in so many Republican primaries. The National Journal's Campaign Tracker shows that more than two-thirds of the Republican delegates up for grabs tomorrow will be awarded on a winner-take-all basis either by state or congressional district or some combination of the two. As such, John McCain's roughly twenty-point leads in most of the national surveys, combined with similar margins in the winner-take-all-states in the Northeast (New York, New Jersey, Connecticut and Delaware) and narrower leads elsewhere position him to take a commanding delegate lead tomorrow night. Mitt Romney's hopes, on the other hand, ride on surpassing McCain in states like California and Missouri.

The Democratic contest is obviously much closer, although in some ways the process of selecting delegates is more straightforward. The allotment of delegates is proportional to votes in each congressional district and each state. While the rules may make for some odd outcomes in individual states (see more detailed explanations here and here), the allotment across all states should be a good reflection of the overall votes cast. While winning individual states may have symbolic value in terms of the way the media covers the results, the total delegate counts amassed across all states are what really matter.

So what do the polls tell us about how tomorrow's Democratic contest will translate into delegates? While Barack Obama appears to be gaining support, there are four reasons to be cautious about what the various polls are reporting [about where the race will end up -- see the clarification below]:

1) Polls are of little use in the caucus states. Roughly 13% of the Democratic delegates chosen tomorrow are from six states and one territory that hold party caucuses (Alaska, Colorado, Idaho, Kansas, Minnesota, North Dakota and American Samoa). Accurate polling in these contests is next to impossible because past turnout has been so light. Fewer than one percent of the eligible adults in the six states participated in the Democratic caucuses in 2004 (ranging from 0.1% in Alaska to 2.2% in North Dakota).

Turnout in the February 5 caucuses is anyone's guess, and as such, pollsters have wisely stayed away. We have logged only two polls in the six caucus states fielded since December. One of these was the Minnesota Public Radio News/Humphrey Institute poll (pdf) that explicitly warned it was "not a prediction of Tuesday night's precinct caucuses" because "the interviews did not identify likely caucus participants." The second, a Mason-Dixon survey in Colorado, is now nearly two weeks old and gave no indication what percentage of Colorado adults were deemed "likely caucus goers."

2) National polls may be misleading. Given the proportional allotment of delegates across such a large number of states, the national polls may provide a reasonable assessment of where the race stands. While we have a lot of very recent national polling data showing Barack Obama gaining, we have to remember that the February 5 states may look different than those not holding contests tomorrow.

So far, I have seen only two national surveys attempt to break out results for the February 5 states, and those show contradictory results.

The report released yesterday by the Pew Research Center allows a comparison across their last three surveys of Democrats in the February 5 states to those who will vote in later primaries. In the December and January surveys, Pew showed no significant difference between these two categories of states. Now, however, Obama does slightly (though not quite significantly) better in the February 5 states. Looking at it another way, virtually all of recent Obama's gains on the Pew survey have come from the February 5 states.

02-04 Pew-2-5.png

On the other hand, the new CBS News survey, which shows the national Clinton-Obama contest deadlocked at 41% each for Clinton and Obama, yields the opposite result. The CBS summary reports the following about a similarly small sample of Democratic primary voters:

The picture in the states voting on Super Tuesday is not nearly as close as the overall picture and offers some good news for Clinton. Among voters in those states, she leads Obama, 49 percent to 31 percent, with 16 percent still undecided.

As Josh Marshall points out, the entire CBS survey was based on 491 Democratic primary voters, so the subgroup of February 5 state voters may have been as small as 200 interviews.

Perhaps our friends at Gallup, who have interviewed nearly 2,200 Democrats over the last five days, can run a tabulation that helps clarify how the February 5 states compare to the rest of the nation. [Update: They did just that -- details here].

3) Are they sampling truly "likely voters?" Some national surveys, such as ABC/Washington Post and CBS, have reported the results of respondents who describe themselves as likely primary voters. Others, however, have reported on the views of registered voters or adults that identify as Democrats. While turnout is likely to be higher tomorrow than in 2004, the percentage of adults that vote in the Democratic primaries is still likely to be smaller than the percentage represented by most of these national surveys.

Here are two sets of turnout statistics to chew over. First, consider how turnout has increased in the Democratic contests held so far:

02-04 turnout so far.png

As should be obvious, turnout has increased dramatically in all the early states even (or perhaps especially) in states that featured little or no active campaigning by the candidates. If nothing else, this pattern suggests that turnout will exceed 2004 levels in all the February 5 states.

It is still worth considering that past turnout has amounted to a relatively small percentage of eligible adults in each state. The following table shows the turnout levels from 2004 for the February 5 primary states as a percentage of all adults an of eligible adults (as reported by Michael McDonald):

02-04 turnout 2004.png

Here's the main point: Even if Democratic turnout doubles tomorrow as compared to 2004, the percentage of adults participating in the Democratic primaries will still be a fraction of the adults identified as Democrats or Democratic "primary voters" on most national polls. Do the truly "likely" voters look different than all Democratic identifiers? Are the statewide surveys doing a better job of selecting "likely voters" than the national polls? Unfortunately, we can only guess, as only a small handful of the statewide surveys report the percentage of adults that their likely voter samples represent.

4) Uncertainty remains high. If you pay attention to nothing else, remember this: As in New Hampshire, a lot of Democrats are having a hard time deciding between Hillary Clinton and Barack Obama. According to the Pew Research Center, both candidates now receive overwhelmingly positive ratings from Democrats:

  • Clinton: 80% favorable, 15% unfavorable
  • Obama: 76% favorable, 16% unfavorable

Again, as in New Hampshire, voters are expressing considerable uncertainty. In California, for example, both the Mason-Dixon and Rasmussen surveys report 29% of Democrats as either completely undecided or indicating there is still a chance they could change their mind about their preference.

This high degree of uncertainty creates the potential for a volatility that the final tracking polls may not reveal. Many voters will likely carry their sense of indecision into the voting booth, so the news and events of the next 24 hours could prove crucial.

Update: Adam's question in the comments suggests the need for a clarification. I have no doubt that support for Barack Obama has been increasing steadily over the last week. Virtually all of the surveys in all of the states are showing evidence of that trend, and as each pollster measures the same population (however it is defined), those trends are reliable. What I am urging caution about is where Clinton-Obama contest ends up when votes are cast tomorrow. As my AAPOR colleague, Professor Robert Shapiro put it over the weekend, "I would trust the trends but not the magnitude - [it] could be greater or less."

**If you have appreciated the constant flow of updates over the weekend, please post a thank you to the indefatigable Eric Dienstfrey for his exceptionally hard work (and for putting up with a boss who sometimes misspells his name).

Correction: The original version of this post incorrectly identified the CBS News survey as a CBS/New York Times survey.

Super Tuesday Endgame

Topics: 2008 , John McCain , Pollster.com , The 2008 Race


Now that the first "Super" event of the week is over, time to turn to the next one.

We've got a number of polls in several states, fewer polls in others. Here are the trends in the 16 states for which we have polls since January 1.

In the Republican chart below, I've omitted Utah where Romney holds an 84-4 lead over McCain, according to the one Republican poll since January 1. Expanding the scale to include that data point makes it hard to see the differences in all the other states.

See the individual state charts at Pollster.com for trend estimates in the states with sufficient polling. Here, look at the data and reach your own conclusions. The data here are through Sunday. I'll update the charts Monday evening with the Monday polls.


Cross-posted at Political Arithmetik.

POLL: CBS National, Rasmussen CA, Suffolk CA, InsiderAdvantage GA/TN, NMSU NM

CBS News
(story, GOP results, Dem results)

Clinton 41, Obama 41... McCain 46, Romney 23

Rasmussen Reports

Obama 45, Clinton 44

Suffolk University (release)

Obama 40, Clinton 39

Insider Advantage

Romney 30, McCain 29, Huck 28... Obama 51, Clinton 36

McCain 32, Huck 30, Romney 22... Clinton 55, Obama 35

New Mexico State University

New Mexico
Obama 48, Clinton 42... McCain 44, Romney 20

Pro-Clinton Push Poll?

Topics: 2008 , Hillary Clinton , John Edwards , LA Times , Pollster , Push "Polls" , The 2008 Race

Andrew Malcolm of the Los Angeles Times reports evidence of a pro-Clinton "push poll" in California, or as he defines it, "malicious political virus that is designed not to elicit answers but to spread positive information about one candidate and negative information about all others under the guise of an honest poll."

His definition is right, but does the call in question meet it? Malcolm's source, a former local television news director named Ed Coghlan, describes a call from "a pollster who wanted to ask registered independents like Coghlan a few questions about the presidential race." The survey tested reactions to statements about Hillary Clinton and negative statements about Barack Obama, John Edwards and John McCain:

Coghlan said he was offended by such underhanded tactics and knew he was going to get out a warning about this dirty trick, but he said he played along for the full 20-minute "poll."

That last bit of information tells me that this call was almost certainly a message testing survey, and not a so-called "push poll." California has over 15 million registered voters, and roughly three million of those are independents. If "someone" was paying "to spread this material phone call by phone call among independent voters," would they really spend 20 minutes on the telephone with each one?

Not likely.

The call that Coghlan describes sounds more like a message testing survey that included many negative messages about Clinton's opponents. In other words, someone called a random sample of voters with the intent to "elicit answers," or more specifically reactions, to negative messages that the Clinton campaign or an allied group considered airing in California.

Negative campaign messages may be offensive, unfair or untrue, and it would certainly be reasonable to question the Clinton campaign on the fairness or truthfulness of the messages tested in this call. Legitimate message testing surveys sometimes cross ethical lines, especially when they raise explosive topics that candidates are unwilling to discuss openly (see the controversy over the calls in New Hampshire and Iowa that tested negative messages about the Mitt Romney's religion).

In this case, however, the only specific negative message that Coughlin reports is the attack on Barack Obama for his "present" votes in the Illinois legislature. Both Clinton and Edwards raised that issue openly in the South Carolina debate.

So far, at least, Malcolm's claim to have uncovered a "malicious political virus" operating "under the guise of an honest poll" is not supported by the facts reported.

For further reading: We have discussed the distinction between so-called "push polls" and message testing many times. Most relevant are my comments on the distinction between "push polls" and message testing here and here a well as those by Stu Rothenberg, Republican pollster Neil Newhouse and the statement from the American Association for Public Opinion Research (AAPOR).

POLL: Rasmussen Georgia, Deseret Utah

Rasmussen Reports

Obama 52, Clinton 37... McCain 31, Romney 29, Huck 28


Deseret Morning News/Dan Jones & Associates

Obama 53, Clinton 29... Romney 84, McCain 4

POLL: Gallup Daily Tracking & Pew National Primary

Gallup Poll

Clinton 46, Obama 44... McCain 43, Romney 24

Note: A separate USA Today/Gallup national survey finds:
Clinton 45, Obama 44... McCain 42, Romney 24


Clinton 46, Obama 38... McCain 42, Romney 22, Huckabee 20

POLL: Tulsa World OK, Rasmussen AZ

Tulsa World/Soonerpoll.com

Clinton 41, Obama 17... McCain 40, 19 Huckabee, 17 Romney

Rasmussen Reports

Clinton 45, Obama 39**... McCain 43, Romney 34

** from cross-tabs (subscription only)

POLL: MSNBC-McClatchy/ Mason-Dixon Primaries

MSNBC-McClatchy/Mason-Dixon (story)

Clinton 47, Obama 41... McCain 37, Huckabee 27, Romney 24

Obama 47, Clinton 41... McCain 33, Romney 27

Clinton 45, Obama 36... McCain 40, Romney 31

New Jersey
Clinton 46, Obama 39... McCain 46, Romney 31

Clinton 43, Obama 41

POLL: Field Poll CA

Field Poll

Clinton 36, Obama 34... McCain 32, Romney 24

POLL: Reuters/C-SPAN/Zogby Primaries


New Jersey
McCain 54, Romney 23... Clinton 43, Obama 42

New York
McCain 49, Romney 23

Obama 48, Clinton 28

McCain 36, Huckabee 27, Romney 22... Clinton 44, Obama 43

Romney 37, McCain 34... Obama 45, Clinton 41

POLL: Monmouth NJ Primary

Monmouth University/Gannett

New Jersey
Clinton 50, Obama 36... McCain 55, Romney 23

POLL: ABC/Post National Survey

ABC News/Washington Post
(ABC story, results; Post story, results)

Clinton 47, Obama 43... McCain 48, Romney 24