Pollster.com

January 6, 2008 - January 12, 2008

 

POLL: CNN National Primary


A new CNN/Opinion Research national survey (story, results) of 1,033 adults (conducted 1/9 through 1/10) finds:

  • Among 397 registered Republicans, Sen. John McCain leads former Gov. Mike Huckabee (34% to 21%) in a national primary; former Mayor Rudy Giuliani also trails McCain at 18%, former Gov. Mitt Romney at 14%, former Sen. Fred Thompson at 6%, Rep. Ron Paul at 5%.
  • Among 443 registered Democrats, Sen. Hillary Clinton leads Sen. Barack Obama (49% to 36%) in a national primary; former Sen. John Edwards trails at 12%.
  • All other candidates receive less than five percent each. The margin of sampling error is 5% for registered Republicans and 4.5% for registered Democrats.


But What About an "Unbounce?"

Topics: 2008 , New Hampshire , The 2008 Race

One of the theories raised to explain the problematic New Hampshire primary polls is a late shift, perhaps on the heels of the Hillary Clinton "tears" story, that polls missed either because they stopped interviewing or completed the bulk of their calls on Sunday or earlier. Two prominent pollsters say such as shift is unlikely given one result in the exit polls.

For example, Andrew Kohut, writing in his must-read op-ed in yesterday's New York Times, concludes:

Yes, according to exit polls the 17 percent of voters who said they made their decision on Election Day chose Mrs. Clinton a little more than those who decided in the past two or three weeks. But the margin was very small — 39 percent of the late deciders went for Mrs. Clinton and 36 percent went for Mr. Obama. This gap is obviously too narrow to explain the wide lead for Mr. Obama that kept showing up in pre-election polls.

Gary Langer, director of Polling for ABC News, agrees in a lengthy review posted this morning:

[T]he exit poll asked voters the time of their decision. Seventeen percent said they decided on Election Day; they voted for Clinton over Obama by a 3-point margin, 39 to 36 percent – hardly a significant swing from the overall result (Clinton +2). Those who said they decided in the previous three days, 21 percent, favored Obama over Clinton by 3 points, 37-34 percent – further deflating the late-decider argument. Those who decided previously, 61 percent of voters, favored Clinton over Obama by 41-37 percent.

For reference, here is the exit poll data that Kohut and Langer cite:

01-11 2008 time of decision.png

But wait. Putting aside the issue of whether respondents can accurately recall when they reached a decision, why does the absence of a big Hillary bump among the late deciders rule out the possibility of some late shift away from Obama? Keeping in mind where Clinton stood in polls before Iowa, we would not be looking for a Clinton bump as much as an Obama "unbump" (to paraphrase the comment left yesterday by my friend reader Mark Lindeman).

Let me explain. Start by looking at our chart of the polls conducted in New Hampshire during 2007. Our trend estimate shows Clinton winning between 35% and 40% of voters between June and early November. As always some individual polls were a little higher, some lower. Her support declined slightly in December, to an average of about 32%, with the usual variation slightly higher and lower. In December, the undecided category averaged 11% and the support for Richardson, , Biden, Dodd and Kucinich was roughly twice what they received on Election Day. Give Clinton a proportional share of the undecideds and those that moved away from the single digit candidates, and she was headed to roughly 37% of the vote. Thus, putting aside whatever happened in the weekend after Iowa, Clinton would not have needed any massive last minute gains to get to the 39% she received in the final count.

 NHTopzDems400m0111.png

Now on the other hand, if Obama had surged after Iowa to a double digit win as the polls seemed to predict, we certainly would have expected to see late deciders favoring him heavily. But if that weekend bump collapsed (or if it never existed in the first place)? In that case, we would not expect to see much of a difference between early and late deciders. Clinton needed only a modest increase in support to reach 39% of the vote.

Consider a historical example. In 1984, Gary Hart surged to victory along a similar trajectory as what polls seemed to forecast for Obama last week. According to the ABC News/Washington Post polls compiled by Samuel Popkin in his book, The Reasoning Voter, Hart trailed Mondale 13% to 37% just before the Iowa Caucuses (with John Glenn running second (at 20%). The Iowa Caucuses were held on a Monday in 1984, a full week before the New Hampshire primary. On interviews conducted Wednesday through Friday after Iowa, Mondale's numbers held constant (38%) while Hart moved up (to 24%). Over the course of the week, Mondale steadly lost support while Hart continued to rise until moving ahead on the Monday before the primary by an eight point margin (35% to 27% -- although ABC reported a three-day rolling average at the time showing the candidates tied). The next day, Hart defeated Mondale, 37% to 28%.

The ABC exit poll also asked voters when they made up their mind, and the pattern is what we would expect. Among those who made up their minds over the final weekend or the two days just before the primary, Hart led Mondale by a four-to-one margin (56% to 14%).

01-11 1984 time of decision.png

So let's consider a hypothetical question: What would have happened if late deciders had broken for Barack Obama by the same margins as they preferred Hart to Mondale in 1984? If I go to my spreadsheet, play "what if" and imagine that late deciders -- those who made up their minds over the last three days -- had preferred Obama 53% to 20% (while all other preferences held constant), Obama would have defeated Hillary Clinton by 10 points (43% to 33%).

Of course, none of this explains exactly what happened last week, and the time-of-decision exit poll data is just one small piece of the puzzle. Opinion polls may have accurately measured a post-Iowa bounce for Barack Obama that "unbounced" over the last 24 hours, or the apparent surge may have been an artifact of some survey error (or perhaps some combination of both). But either way, the lack of a difference between late and early deciders does not tell us much. It certainly does not preclude -- by itself -- the possibility of shift of voters to Obama on Saturday and Sunday that shifted back to Clinton on Monday.


POLL: SurveyUSA New York Primary


A new SurveyUSA automated survey of likely primary voters in New York State (conducted 1/9 through 1/10) finds:

  • Among 471 likely Republican primary voters, former Mayor Rudy Giuliani runs at 32%, Sen. John McCain at 29% in a statewide primary; former Gov. Mike Huckabee trails at 12%, former Gov. Mitt Romney at 7%, former Sen. Fred Thompson at 6%.
  • Among 957 likely Democratic primary voters, Sen. Hillary Clinton leads Sen. Barack Obama (56% to 29%) in a statewide primary; former Sen. John Edwards trails at 8%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4.6 for likely Republican primary voters and 3.2 for likely Democratic primary voters.


NH: A Lesson From 1948

Topics: 2008 , New Hampshire , The 2008 Race

My second** NationalJournal.com column (that like all these contributions will be free to non-subscribers for the next week): The lesson from the polling debacle of 1948 that pollsters should apply in aftermath of this week's polling problems in New Hampshire.

**PS: Second? It's been busy week. Details on the new column here.



Our New Partnership with National Journal Group

Topics: Pollster.com

This news has been a bit delayed between the holidays and the early primaries, but we are very excited to announce a new strategic partnership between Pollster.com and the National Journal Group, the publishers of the National Journal, CongressDaily, The Hotline, The Almanac of American Politics, and NationalJournal.com. Pollster.com will remain an independent website, just as it is now, but we will write a column on NationalJournal.com and provide content from Pollster.com to the various in-house polling resources within National Journal Group publications. In addition, Charles Franklin and I will write a column and provide features that will appear on NationalJournal.com.

What has us especially excited here is the opportunity to work with the top notch journalists that are a part of the National Journal Group and it's parent, Atlantic Media, including my old friend Amy Walter and her colleagues at The Hotline Ron Brownstein of NationalJournal.com and Charlie Cook's Political Report. The move of our "world headquarters" that I hinted at was to the National Journal Group offices in Washington, which also puts us just a few floors away from our friends at The Atlantic, including Andrew Sullivan and Marc Ambinder.

My first column -- some thoughts about what the success of the Des Moines Register Poll says about the philosophy of behind likely voter models -- appeared earlier this week (and should still be free to non-subscribers).

And as long as we are in a moment of transition, I want to say a big public thank you to both Charles Franklin and Eric Dienstfrey, who of course, are very much involved in this new partnership. If you like the charts and graphics here at Pollster, all credit goes to Charles, especially since his contribution here comes on top of a demanding day job at the University of Wisconsin. Second, Eric Dienstfrey, the hardest working man in show business (or at least at Pollster.com), has been working more or less non-stop for the last few weeks updating our charts and tables and the poll updates on the blog. We are all a bit fried at this point and yes, we have missed a typo or two in recent weeks, but if you appreciate all this site has to offer, you might want to leave Eric and Charles a big thank you in the comments section below.

Finally, a huge continuing thank you to Doug Rivers and YouGov/Polimetrix, for their sponsorship and unwavering commitment to our editorial independence without which Pollster.com would not exist.

It is going to be a long, interesting year and we are looking forward to helping you make sense of polls and polling data the whole way through.


POLL: FOX South Carolina Republican Primary


A new FOX News/Opinion Dynamics statewide survey (story, results) of 500 likely Republican primary voters in South Carolina (conducted 1/9) finds:

  • Sen. John McCain runs at 25%, former Gov. Mike Huckabee at 18%, former Gov. Mitt Romney at 17% in a statewide primary; former Sen. Fred Thompson trails at 9%, former Mayor Rudy Giuliani and Rep. Ron Paul both at 5%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4%.
  • "Respondents were randomly selected from two source: 250 were randomly selected from a list of voters who had previously voted in a South Carolina Republican primary, and 250 were drawn from a random digit dial sample which gives every household in the state an equal chance of being called."


Women Non-Working For Hillary

Topics: General , Hillary Clinton

[Margie Omero is President of Momentum Analysis, a Democratic polling firm based in Washington, DC.]

The big news this week is that women, who voted for Obama in Iowa, put Hillary Clinton over the edge in New Hampshire. But rather than treat women as a monolithic group, it's worth examining the exit polls closely to look at the role of work status, parental status, and marital status in the New Hampshire vote. Despite the increasing focus on women's marital status, in addition to the usual focus on parental status (such as the "security moms" and "soccer moms" of yore), it is actually women not working full-time who are most likely to vote for Clinton.

Exit polls from the Democratic primary helpfully include breakouts by marital status by gender, by parental status by gender, as well as breakouts among married women with children, and women who work full-time. I extrapolated the rest (identified with an asterisk), and include it all in the table below.

(Note: The exit polls defines "parents" as the presence of children under 18 in one's home, and unmarried is not broken out further. And probably because of different versions of the exit poll questionnaire, Obama receives 32% of the vote from both married and unmarried women, but receives 34% from women overall.)

mo0110.png

A few patterns emerge:

  • There is no real difference in Obama's edge between men with kids at home and men without kids at home.
  • But among men, there is a sizable difference by marital status. Obama has a much larger lead with unmarried men than with married men.**
  • Among women, kids at home makes more of a difference than marital status. Those without kids at home are more likely to support Clinton than those with kids at home, across marital status groups. Unmarried women without kids at home are most likely to support Clinton, while married women with kids at home are least so.
  • All groups of women we can examine with the exit polls give Clinton an edge.

But the most salient difference by gender is among women not working full-time. Clinton receives a strong 25-point lead with this group, compared to her 3-point lead among those who do work full-time. And while this could be partly due to older retired women being in the non-working group, it's likely socioeconomic status plays a large role, too.

In fact, aside from voters without a high school diploma, no other demographic group gave Clinton such a large margin. (I'm not counting "favorable toward Clinton" or prioritizing "right experience" as demographic groups.) Clinton also had a stronger lead with voters earning under $50,000 a year, with those who feel the country's economy is poor, and with those who say the economy is the most important issue. The table below shows her standing with voters at the lower end of the socioeconomic spectrum.

mo01102.png

Because without a dataset-or even crosstabs-we can't discern causation from the exit polls, it's worth considering the different options. Much has been made about the role of gender in the New Hampshire primary. Everything from the "diner sob" to aggrieved women fed up with sexism to Gloria Steinem's NYT op-ed piece has been credited. But perhaps causality went the other direction. Maybe a long-standing Clinton infrastructure in New Hampshire grounded her base with lower socioeconomic folks, who happen to be more likely to be women. She has done well with this group for some time, and did best (+17 over Obama) with voters who decided earlier than a month ago.

Why is it because women gave Clinton the edge, we assume it's because they had an emotional reaction to Clinton (and her gender) personally? Clinton did better with those who said the "candidate's position on issues" was most important (39% Clinton, 34% Obama) than with those who said the "candidate's leadership/personal qualities" were most important (37% Clinton, 45% Obama). It might be that Clinton's female support may have been actually considering issues like the economy, rather than listening to their emotions.

** typo corrected.


POLL: Strategic Vision (R) Michigan Republican Primary


A new Strategic Vision (R) statewide survey of 700 likely Republican primary voters in Michigan (conducted 1/4 through 1/6) finds Sen. John McCain leads former Gov. Mitt Romney (29% to 20%) in a statewide primary; former Gov. Mike Huckabee also trails McCain at 18%, former Mayor Rudy Giuliani at 13%, former Sen. Fred Thompson and Rep. Ron Paul both trail at 5%.

All other candidates receive less than five percent each. The margin of sampling error is 4%.


POLL: Rossman/MIRS/Denno-Noor (D) MI Primary


A new Rossman Group/MIRS/Denno-Noor (D) statewide survey of of likely primary voters in Michigan (conducted 1/6 through 1/7) finds:

  • Among 300 likely Republican primary voters, former Gov. Mike Huckabee runs at 23%, former Gov. Mitt Romney at 22%, Sen. John McCain at 18% in a statewide primary; former Mayor Rudy Giuliani trails at 8%.
  • Among 300 likely Democratic primary voters asked to choose from "a Democratic field that does not include Barack Obama or John Edwards," Sen. Hillary Clinton runs at 48% in a statewide primary; "uncommitted" at 28%, "unsure" at 11%, and "other" and 10%.
  • All other candidates receive less than five percent each. The margin of sampling error is 5.8% for each subgroup.


POLL: InsiderAdvantage FL, SC Primaries


Two new InsiderAdvantage statewide surveys of likely primary voters in Florida and South Carolina (conducted 1/7) finds:

  • Among 393 likely Democratic primary voters in South Carolina, Sen. Barack Obama runs at 40%, Sen. Hillary Clinton at 33%, former Sen. John Edwards at 15%. Among 197 African Americans (50% of the sample), Obama receives 48%, Clinton 37%, Edwards 6%.
  • Among 479 likely Republican primary voters in South Carolina, former Gov. Mike Huckabee leads Sen. John McCain (33% to 21%) in a statewide primary, former Gov. Mitt Romney trails at 14%, former Mayor Rudy Giuliani at 8%, former Sen. Fred Thompson and Rep. Ron Paul at 5%.
  • Among 303 likely Democratic primary voters in Florida, Clinton runs at 40%, Obama at 32%, Edwards at 9%, Richardson at 6%. Among 340 Republicans, Giuliani runs at 24%, Huckabee and McCaina t 19%, Romney at 13%, Thompson at 8%, Paul at 5%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4% for likely Republican primary voters in South Carolina, 5% for likely Democratic primary voters in South Carolina, 5.5% for likely Republican primary voters in Florida, and 6% for likely Democratic primary voters in Florida.


New Hampshire: So What Happened?

Topics: 2008 , New Hampshire , The 2008 Race

There is obviously one and only one topic on the minds of those who follow polls today. What happened in New Hampshire? Why did every poll fail to predict Hillary Clinton's victory?

Let's begin by acknowledging the obvious. There is a problem here. Even if the discrepancy between the last polls and the results turns out to be about a big last minute shift to Hillary Clinton that the polls somehow missed (and that certainly sounds like a strong possibility), just about every consumer of the polling data got the impression that a Barack Obama victory was inevitable. One way or another, that's a problem.

For the best summary of the error itself, I highly recommend the graphics and summary Charles Franklin posted earlier today. Here's a highlight of how the result compared to our trend estimates:

What we see for the Democrats is quite stunning. The polls actually spread very evenly around the actual Obama vote. Whatever went wrong, it was NOT an overestimate of Obama's support. The standard trend estimate for Obama was 36.7%, the sensitive estimate was 39.0% and the last five poll average was 38.4%, all reasonably close to his actual 36.4%.

It is the Clinton vote that was massively underestimated . . .Clinton's trend estimate was 30.4%, with the sensitive estimate even worse at 29.9% and the 5 poll average at 31.0% compared to her actual vote of 39.1%.

So what went wrong? We certainly have no shortage of theories. See Ambinder, Halperin, Kaus, and, for the conspiratorially minded, Friedman. The pollsters that have weighed in so far (that I've seen at least) are ABC's Gary Langer (also on video), Gallup's Frank Newport, Scott Rasmussen and John Zogby. Also, Nancy Mathiowetz, president of the American Association for Public Opinion Research (AAPOR) has blogged her thoughts on Huffington Post.

Figuring out what happened and sorting through the possibilities is obviously a much bigger task than one blog post the morning after the election. But let me quickly review some of the more plausible or widely repeated theories and review what hard evidence we have, for the moment, regarding each.

1) A last minute shift? - Perhaps the polls had things about "right" as of the rolling snapshot taken from Saturday to Monday, but missed a final swing to Hillary Clinton that occurred over the last 24 hours and even as voters made their final decisions in the voting booth. After all, we knew that a big chunk of the Democratic electorate remained uncertain and conflicted, with strong positive impressions of all three Democratic front-runners. The final CNN/WMUR/UNH poll showed 21% of the Democrats "still trying to decide" which candidate they would support, and the exit poll showed 17% reported deciding on Election Day with another 21% deciding within the last three days. Polls showed Clinton polling in the mid to upper 30s during the late fall and early winter before a decline in December. Perhaps some supporters simply came home in the final hours of the campaign.

I did a quick comparison late last night of the crosstabs from the exit polls and final CNN/WMUR/UNH survey. Clinton's gains looked greatest among women and college educated voters. That pattern, if it also holds for other polls (a big if) seems suggestive of a late shift tied to the intense focus on Clinton's passionate and emotional remarks, especially over the last 24 hours of the campaign.

2) Too Many Independents? - One popular theory is that polls over-sampled independent voters who ultimately opted for a Republican ballot to vote for John McCain. I have not yet seen any hard turnout data on independents from the New Hampshire Secretary of State, but the exit poll data does not offer promising data for this theory. As I blogged yesterday, final Democratic polls put the percentage of registered independents (technically "undeclared" voters) at between 26% and 44% (on four polls that released the results of a party registration question). The exit poll reported the registered independent number as 42%, with another 6% reporting they were new registrants. So if anything polls may have had the independent share among Democrats too high.

On Republican samples, pre-election pollsters reported the registered independent numbers ranging between 21% and 34%. The exit poll put it at 34%, with 5% previously unregistered. So here too, the percentage of independents may have been too low.

Apply those percentages to the actual turnout, do a little math, and you get an estimate of how the undeclared voters split: roughly 60% took a Democratic ballot and 40% a Republican. That is precisely the split that CNN/WMUR/UNH found in their last poll although other

Keep in mind that the overall turnout was over 526,671 (or 53.3% of eligible adults). Eight years ago (the last time both parties had contested primaries) it was 396,385 (or 44.4% of eligible adults at the time). That helps explain why we may have seen an increase in independents in both parties.

Of course, we are missing a lot of data here: Nothing yet on undeclared voter participation from the Secretary of State, and roughly half the pollsters never released a result for party registration.

3) Wrong Likely Voters? OK, so maybe they had the independent share right, but perhaps pollsters still sampled the wrong "likely voters" by some other measure. The turnout above means that pollsters had to try to select (or model) a likely electorate that amounted to roughly half the adults in New Hampshire, they reached with a random digit dial sample.

Getting the right mix is always challenging, possibly more so because the Democratic turnout was so much higher than in previous elections. That's an argument blogged today by Allan McCutcheon of Edison Research:

In 2004, a (then) record of 219,787 voters turned out to vote--the previous record for the Democratic primary was in 1992, when 167, 819 voters participated. This year, a record shattering 287,849 voters participated in the New Hampshire Democratic primary--including nearly two thirds (66.3%) of the state's registered Democrats (up from 43.3% in 2004). Simply stated, the 2008 New Hampshire Democratic primary had a voter turnout rate that resembled a November presidential election, not a usual party primary, and the likely voter models for the polling organizations were focused on a primary--this time, that simply did not work.

One way to assess whether polls sampled the wrong kinds of voters would be to look carefully at their demographics (gender, age, education, region) and see how they compared to the exit poll and vote return data. Unfortunately, as is so often the case, only a handful of New Hampshire pollsters reported demographic composition.

4) The Bradley/Wilder effect? The term, as wikipedia tells us, derives from the 1982 gubernatorial campaign of Tom Bradley, then the long time African-American mayor of Los Angeles. Bradley led in pre-election polls but lost narrowly. A similar effect, in which polls understated the support for the opponents of African-American candidates seemed to hold in various instances during the 1980s. Consider this summary of polls compiled by the Pew Research Center for a 1998 report: which they updated in February 2007:

nhmark0109.png

Note that, in almost every instance, the polls were generally about right in the percentage estimate for African-American candidate but tended to underestimate the percentage won by their white opponents. The theory is that some respondents are reluctant to share an opinion that might create "social discomfort" between the respondent and the interviewer, such as telling a stranger on the telephone that you intend to oppose an African-American candidate.

Of course, the Pew Center also looked at six races for Senate and Governor in 2006 that featured an African-American candidate and did not see a similar effect. Also keep in mind that that all of the reports mentioned above that show the effect were from general election contests, not primaries.

What other evidence might suggest the Bradley/Wilder effect operating in New Hampshire in 2008? We might want to consider whether the race of interviewer or the use of an automated (interviewer-free) methodology would have an effect, although these kinds of analyses are difficult, because other variables can confound the analysis. For what it's worth, the final Rasmussen automated survey had Obama leading by seven points (37% to 30%), roughly the same margin as the other pollsters. We might also look at whether pushing undecided voters harder helped Clinton more than other candidates.

Update: My colleagues at AAPOR have made three relevant articles from Public Opinion Quarterly available to non-subscribers on the AAPOR web site.

5) Non-response bias? We would be crazy to rule it out, since even the best surveys are getting response rates in the low twenty percent range. If Clinton supporters were less willing to be interviewed last weekend than Obama supporters, it might contribute to the error. Unfortunately, it is next to impossible to investigate, since we have little or no data on the non-respondents. However, if pollsters were willing to be completely transparent, we might compare the results among those with relatively high response rates to those with lower rates. We might also check to see if response rates declined significantly over the final weekend.

6) Ballot Placement? Gary Langer's review points to a theory offered by University Prof. Jon Krosnick, that Clinton's placement near the top of the New Hampshire ballot boosted her vote total. Krosnick believes that ballot order netted Clinton "at least 3 percent more votes than Obama."

7) Weekend Interviewing? I blogged my concerns on Sunday. Hard data on whether this might be a factor are difficult to come by, but it is certainly an issue worth pursuing.

8) Fraud? As Marc Ambinder puts it, some are ready to believe "[t]here was a conspiracy, somehow, because pre-election polls are just so much more valid than actual vote counts." Put me down as dubious, but Brad Friedman's Brad Blog has the relevant Diebold connections for those who are interested.

Again, no one should interpret any of the above as the last word on what happened in New Hampshire. Most of these theories deserve more scrutiny and I agree with Gary Langer that "it is incumbent on us - and particularly on the producers of the New Hampshire pre-election polls - to look at the data, and to look closely, and to do it without prejudging." This is just a quick review, offering what information is most easily accessible. I am certain I will have more to say about this in coming days.


Polling Errors in New Hampshire

Topics: 2008 , New Hampshire , The 2008 Race

1NHPollErrorDem19.png

Hillary Clinton's stunning win over Barack Obama in New Hampshire is not only sure to be a legendary comeback but equally sure to become a standard example of polls picking the wrong winner. By a lot.

There is a ton of commentary already out on this, and much more to come. Here I simply want to illustrate the nature of the poll errors. These show the nature of the problem and help clarify the issues. I'll be back later with some analysis of these errors, but for now let's just see the data.

In the chart, the "cross-hairs" mark the outcome of the race, 39.1% Clinton, 36.4% Obama. This is the "target" the pollsters were shooting for.

The "rings" mark 5%, 10% and 15% errors. Normal sampling error would put a scatter of points inside the "5-ring", if everything else were perfect.

In fact, most polling shoots low and to the left, though often within or near the 5-ring. The reason is undecided voters in the survey. Unless the survey organization "allocates" these voters by estimating a vote for them, some 3-10% in a typical election survey are left out of the final vote estimate. Some measures of survey accuracy divide the undecided, either evenly across candidates or proportionately across them. There is good reason to do that in another post. But what the pollsters publish are the unallocated numbers (almost always) and so it seems fair to plot here the percent of the vote the pollster published, not one with undecided reallocated.

What we see for the Democrats is quite stunning. The polls actually spread very evenly around the actual Obama vote. Whatever went wrong, it was NOT an overestimate of Obama's support. The standard trend estimate for Obama was 36.7%, the sensitive estimate was 39.0% and the last five poll average was 38.4%, all reasonably close to his actual 36.4%.

It is the Clinton vote that was massively underestimated. Every New Hampshire poll was outside the 5-Ring. Clinton's trend estimate was 30.4%, with the sensitive estimate even worse at 29.9% and the 5 poll average at 31.0% compared to her actual vote of 39.1%.

So the clear puzzle that needs to be addressed is whether Clinton won on turnout (or Obama's was low) or whether last minute decisions broke overwhelmingly for Clinton. Or whether the pollster's likely voter screens mis-estimated the make up of the electorate. Or if the weekend hype led to a feeding frenzy of media coverage that was very favorable to Obama and very negative towards Clinton, which depressed her support in the polls but oddly did not lower her actual vote.

On the Republican side we see a more typical pattern, and with better overall results. About half of the post-Iowa polls were within the 5-ring for the Republicans, and most of the rest within the 10-ring.

2NHPollErrorRep19.png

As expected, errors tend to be low and left, but the overall accuracy is not bad. This fact adds to the puzzle in an important way:

If the polls were systematically flawed methodologically, then we'd expect similar errors with both parties. Almost all the pollsters did simultaneous Democratic and Republican polls, with the same interviewers using the same questions with the only difference being screening for which primary a voter would participate in. So if the turnout model was bad for the Democrats, why wasn't it also bad for the Republicans? If the demographics were "off" for the Dems, why not for the Reps?

This is the best reason to think that the failure of polling in New Hampshire was tied to swiftly changing politics rather than to failures of methodology. However, we can't know until much more analysis is done, and more data about the polls themselves become available.

A good starting point would be for each New Hampshire pollster to release their demographic and cross tab data. This would allow sample composition to be compared and for voter preferences within demographic groups to be compared. Another valuable bit of information would be voter preference by day of interview.

In 1948 the polling industry suffered its worst failure when confidently predicting Truman's defeat. In the wake of that polling disaster, the profession responded positively by appointing a review committee which produced a book-length report on what went wrong, how it could have been avoided and what "best practices" should be adopted. The polling profession was much the better for that examination and report.

The New Hampshire results are not on the same level of embarrassment as 1948, but they do represent a moment when the profession could respond positively by releasing the kind of data that will allow an open assessment of methods. Such an assessment may reveal that in fact the polls were pretty good, but the politics just changed dramatically on election day. Or the facts could show that pollsters need to improve some of their practices and methods. Pollsters have legitimate proprietary interests to protect, but big mistakes like New Hampshire mean there are times when some openness can buy back lost credibility.

Cross-posted at Political Arithmetik.


POLL: Datamar Florida Primary


A new Datamar automated survey of likely primary voters in Florida (conducted 1/5 through 1/7) finds:

  • Among 541 likely Democratic primary voters, Sen. Hillary Clinton leads Sen. Barack Obama (40% to 28%) in a statewide primary; former Sen. JOhn Edwards trails at 19%.
  • Among 481 likely Republican primary voters, former Gov. Mike Huckabee runs at 24%, former Gov. Mitt Romney at 20%, Sen. John McCain at 18%, former Mayor Rudy Giuliani at 16%, former Sen. Fred Thompson at 9%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4.2% for likely Democratic primary voters and 4.5% for likely Republican primary voters.


ABC News Exit Poll Analysis


New ABC News analysis looks at the New Hampshire primary exit poll data and finds:

  • "The economy was the single biggest issue to voters in both parties, and that worked for Clinton as well. Among those who called it their top concern, she beat Obama by 9 points, 44-35 percent. Clinton also won lower income and lower-education voters..."
  • "Clinton led among those who chose a candidate early, while Obama's support was greatest -- 43 percent to Clinton's 28 percent - among people who decided which candidate to support sometime last week. Clinton became somewhat more competitive among those who decided more recently."

Read the full analysis here.


NH Results Thread

Topics: 2008 , Exit Polls , New Hampshire , The 2008 Race

6:08 p.m. Eastern time - The Atlantic's Marc Abminder is first out of the box with hints from the New Hampshire exit poll:

EXIT POLLS: GOP: 3 in 10 independents are GOP voters...many late deciders... McCain more electable than Romney...33% say economy is biggest issue followed by Iraq (22%) .... Democrats: 46% made up minds without last week.. 4 in 10 are independents.... HRC's favorability: 73%; Obama's: 84%; ... 36% say economy is top issue....

MSNBC is also reporting that nearly half of the Democrats voting in New Hampshire are independents (43%) and that 38% of the Republicans are independent. Those numbers within range but on the high side of what pre-election polls were reporting this week for both parties, although the Democratic-Republican split is more favorable to the Democrats than the CNN/WMUR/UNH and Fox News polls reported. See the post I updated just a few minutes ago for more details.

I am headed home...but the comments section is open, please post your questions. Apologies if you get the dreaded "too many comments" error -- you did nothing wrong. We will squash that bug soon.

7:25 - Via Ben Smith a this report from ABC News posted at 6:00:

ABC News' Gary Langer Reports: Based on preliminary exit poll results from the New Hampshire primaries, Independents are turning out in substantial but customary numbers.

Preliminary exit poll results indicate that just over four in 10 voters in the New Hampshire Democratic primary are independents, compared with 48 percent in 2004 and a record 50 percent in 1992.

To be clear, the numbers from 2004 and 1992 that Langer cites are of party identification. The numbers cited by MSNBC above may be as well. The numbers I reported earlier from pre-election surveys are mostly party registration. And there is a difference, but I have no idea what that difference might imply.

7:50 - Exit poll tabulations will be posted on MSNBC at these links (Democrats, Republicans) when the polls close at 8 p.m. Eastern time. CNN will presumably post tabulations as well. See my post from earlier this morning for more information on what to make of these numbers when they appear.

8:04 - CNN has tabulations up for Democrats and Republicans.

8:11 - Mark Lindeman is posting extrapolations of the current candidate estimates used the weight the tabulations posted on CNN and MSNBC in the comments section below. Look there for further updates.

8:28 - The tabulations on the Democratic side indicate that Obama's advantage was narrower among those who decided in the last few days than among those who decided over the last month (i.e. no big break to Obama in the closing days): Obama's margin over Clinton (using the current estimates, which will likely change over the next few hours) among those who decided their vote today (40% to 37%), the last three days (41% to 35%), last week (47% to 25%), last month (48% to 32%). Among those who decided "before that," Clinton leads 47% to 32%. And more than a third of voters (37%) say they made up their minds today or in the last three days.

10:33 - NBC projects Clinton the winner, prompting the following exchange between the MSBNC anchors:

Chris Matthews: [Clinton] has beaten the odds, she has beaten the pollsters, the pundits. Everyone one of us included who has been trying to follow this campaign and understand it. I think something happened. It must have happened fairly recently, or else the pollsters should find another means of employment.

Keith Olbermann: Well the entire industry was apparently mistaken, it had nothing to do...

Matthews: But every poll. At least with the other side [the Republicans] there was some disagreement, in the Democratic primary, these polls were relentlessly pro-Obama.

12:35 - As I started to write up these final paragraphs, Chris Matthew popped up to say the following on MSNBC: "I'd like to see an inquest of all these polls and the methodology because we always have learned, eventually, what went wrong with polling."

Well, what follows is considerably less than an inquest, but I have been comparing the exit poll tabulations with the last set of cross-tabulations from CNN/WMUR/UNH. Looking at just one poll may turn out to be misleading, so hopefully we can do similar comparisons on a larger group of polls, but based on this initial look, here is what I see:

If there was a problem with this one poll it was not about the composition of the electorate. Were there too few women? Too many independents? Too many young voters? On these three variables, if it erred, the UNH poll erred slightly in Clinton's favor. It had slightly more women, more older voters and more registered independents in the Democratic electorate than the exit poll. The UNH poll did sample slightly more voters with college degrees (61%) than the exit poll (53%), but that difference does not explain Obama's lead. Weight back 61% college educated to 53%, and Obama's lead on the poll shrinks only a little (from 9 to 6 points).

On the other hand, the discrepancy between the last UNH poll and the result seems concentrated in a few key subgroups. I will post the exact numbers tomorrow once the we get a final exit poll tabulations, but virtually all of the difference seems to come from women and college educated voters. For the moment, when comparing the UNH poll to the exit poll, I see a net 17 point gain for Clinton among women compared to a 5 point gain among men, and a 13 point net gain among college educated voters compared to a one point net loss among those with no college degree.

My new colleague* Ron Brownstein has chronicled the critical importance of college educated women as swing voters in the Democratic nomination race. More than any other group, they moved to Clinton in the fall after her strong performances in early debates. Yes she appeared to be doing far less well among these voters in Iowa. If the polls missed a last minute shift to Clinton in New Hampshire, considering the heavily gender focused coverage of the last 48 hours of the campaign, the most logical place to look is among college educated women.

Combine that with the exit poll results showing 37% of the Democrats "finally deciding" for whom they would vote in the last three days of the campaign, and we have a pretty good first clue of what happened with the polls in New Hampshire this week.

*There is a hint there for regular readers -- more on that tomorrow (er..later today).

12:50 - ABC polling director Gary Langer has some very worthy first impressions:

There will be a serious, critical look at the final pre-election polls in the Democratic presidential primary in New Hampshire; that is essential. It is simply unprecedented for so many polls to have been so wrong. We need to know why.

But we need to know it through careful, empirically based analysis. There will be a lot of claims about what happened - about respondents who reputedly lied, about alleged difficulties polling in biracial contests. That may be so. It also may be a smokescreen - a convenient foil for pollsters who'd rather fault their respondents than own up to other possibilities - such as their own failings in sampling and likely voter modeling. [....]

The data may tell us; it may not. What's beyond question is that it is incumbent on us - and particularly on the producers of the New Hampshire pre-election polls - to look at the data, and to look closely, and to do it without prejudging.

Definitely worth reading in full.


Pre-NH "Outliers"

Topics: 2008 , The 2008 Race

Frank Newport considers the paradox of Mike Huckabee's support rising nationally while he remains in third place in New Hampshire.

Gary Langer argues that the large numbers of independents that participate in the New Hampshire primary "have compromised the state’s predictive power" regarding the primary contests that follow.

Jennifer Agiesta has some very useful cross-tabulations by age from the Iowa entrance poll. Agiesta and Jon Cohen also mined the entrance poll to offer five things to watch in New Hampshire.

Dante Scalia has his own list of things to watch for the Democrats and Republicans in New Hampshire tonight.

The New Hampshire Attorney General names the polling firm that allegedly made the controversial message testing survey that asked questions about Mitt Romney's Mormon faith: Moore Information Systems.

Noam Scheiber has an equation that political scientists will enjoy.

And finally, CafePress offers, not a survey, but an interactive Flash graphic showing how t-shirts for each candidate are selling on their site.


New Hampshire Pre- and Post-Iowa

Topics: 2008 , The 2008 Race

1NHDeltaPlot18.png

Here is one more way to look at the impact of Iowa. The plot shows each poll (tracking polls are included only when their samples don't overlap, e.g. every 3rd day for a 3 day track.) Polls ending on the same day are separated in the chart, though the order is arbitrary within day.

McCain had already captured the lead over Romney by the Iowa caucus. While three post-Iowa polls still find a Romney lead, the majority of polls put McCain in the lead. Moreover, the height of the bars is quite similar before and after Iowa, indicating little effect.

That lack of change is interesting since Romney might have been expected to fall due to his disappointing second place finish in Iowa. McCain's 4th place in Iowa wasn't read as a "loss", in the bizarre expectations game. But regardless of their finishes, Iowa seems to have not moved the New Hampshire Republican electorate at all.

On the Democratic side, the large pro-Obama bounce is obvious. If it didn't show up in the first day or two post-Iowa, it clearly appears after that. A Clinton lead of some 10-15 points became an Obama lead of some 8-10 points.

Cross-posted at Political Arithmetik.


NH Election Day Thoughts

Topics: 2008 , Incumbent Rule , New Hampshire , The 2008 Race

I have few thoughts about New Hampshire in my mental "in-box" I want to try to blog this afternoon...

1) Break to Obama? - Reader "FlyOnTheWall," in a comment posted to my exit poll item this morning, noticed something important in the final round of polls on the Democratic race in New Hampshire:

I was struck by something this morning, looking at the final two days of tracking polls, and was hoping that you could illuminate the issue.

There have been 16 numbers released over the past two days, all conducted entirely since Iowa. Of these polls, 14 have pegged Hillary's support in a very narrow range of 28-31 percent. (The other two are from Suffolk, which has been a consistent outlier throughout primary season, but even Suffolk is only at 34.)

The Obama polls, by contrast, are all over the map. They put him anywhere from 32 to 42 percent, and are fairly evenly distributed over that range. In other words, everyone seems to agree on Hillary's level of support - but what determines the margin is the level of support for Obama.

What gives here?

Several readers have posted responses worth reviewing. Here is my quick take. "Fly" is right about the pattern in the data, as the following table shows. There is less variation in the Clinton percentage than for the other candidates, particularly Obama. But notice that Obama's support generally goes up as the percentage of undecided voters goes down.

01-08 final dems.png

That patterns suggests that as of the final snapshot, a lot of voters are leaning to Obama but not quite yet decided. That pattern is consistent with what we sometimes call the "incumbent rule." Obviously, Senator Clinton is not an incumbent, but much as they often do with incumbent candidates, voters may have largely made their decision about Clinton yet are still in the process of deciding to support her most prominent opponent. The final CNN/WMUR/UNH poll still has 21% of the likely Democratic voters saying they have "considered some candidates but are still trying to decide," including 18% of those who say they will "definitely vote" in the primary. Thus, some voters will carry their uncertainty all the way to their polling place, not making their final decision until they cast a ballot. This pattern usually suggests a "break" to the challenger (or in this case Obama), but as I learned the hard way in 2004, not always. We will know later tonight.

PS: MIckey Kaus and his readers noticed signs of the "incumbent rule" pattern in Iowa.

2) What is the Independent Mix? On First Read this morning, our friend Chuck Todd passed along the following:

Team Romney believes many of the tracking polls could be over-sampling independents. And if the indies move en masse to Obama, it could make for a more conservative GOP electorate, benefiting both Romney and (to a lesser extent) Huckabee.

This observation raises a good question. What is the percentage independent on the various tracking surveys. Unfortunately not all pollsters have included the relevant data in their releases, but the following table shows what I was able to gather from the final surveys:

01-11 nh ind.png

A few notes. First, not all surveys ask about party the same way. Most of the numbers cited above appear to be based on self-reported party identification (what do you "consider yourself?") and party registration ("how are you registered?"). The CBS/New York Times survey, for one, reported party ID results. Second, I certainly may have overlooked party numbers, so please email me or leave a comment if you can fill in party numbers missing above.

Note, the numbers above are not the "independent split" calculation that Noam Scheiber blogs about here, although obviously, those numbers are related. The CNN/WMUR/UNH survey reported a 60% to 40% split to the Democrats on their last survey, Fox reported a 55% to 45% split in the same direction.

Two more points about independents (or more correctly, those whose party registration is "undeclared"). Some data in the CNN/WMUR/UNH report makes it clear that the bigger the turnout, the greater the undeclared contribution in both party primaries. Not surprisingly, as the table below shows, the "definite" voters voters are more likely to be undeclared than those who say they "may vote" or that they plan to vote but not if an "emergency" comes up.

01-07 ind by turnout.png

One indirect measure to look for in the exit poll tabulations tonight is the percentage independent in each primary, and how it compares to what the pre-election polls were reporting.

Note: I want to apologize to those who continue to receive the "too many comments" error message when attempting to post a comment here. Suffice it to say, you have not posted too many comments to Pollster.com. We have been trying to squash this bug for months now (without success) and, unfortunately, the very heavy traffic we are experiencing today is aggravating the bug.


Last Day of the New Hampshire Endgame

Topics: 2008 , The 2008 Race

1NHEndgame18.png

The last New Hampshire polls are in, and there is little change of trend from Monday. The Obama rise continues to look steady and strong. The sensitive trend estimator in red continues fit the sharp turn in Obama support better than the slower-to-change blue estimator, though both end up close to one another in estimating current support.

The current estimates are 39.0% for Obama and 29.9% for Clinton. If you like the more stable standard blue estimate, the numbers are 36.7% and 30.4% respectively.

There is the tiniest of hints that Romney's decline and McCain's surge have both flattened just a bit in the sensitive Red estimator, though the actual difference between the red and blue trend estimates is trivial for each candidate.

The sensitive red trend puts McCain at 33.4% and Romney at 27.5%, while the standard blue estimate has it 34.2% to 27.5%.

The first bit of suspense tonight will be whether McCain succeeds in holding a lead over Romney. The trend data say yes, but there is some considerable variation in the McCain poll results.

On the Democratic side, the question should be the size of the Obama win.

Edwards looks to be a distant third, a result that should be damaging to his campaign to emerge as one of the two "change" candidates.

Huckabee appears to have utterly failed to capitalize on his Iowa success. That is a big deal, in my view, because his campaign needed dramatic successes to bootstrap itself into a national effort. Iowa alone is not enough. Can he recapture the Iowa momentum in Michigan or Nevada or South Carolina in the face of a new look at McCain? (Or a reborn Romney should an upset happen in NH.)

The Thompson campaign gave up in NH, and Giuliani's collapse there just continues the question of whether his once high flying campaign can survive the early series of losses he now seems set for.

Some of these questions will be answered tonight.

Cross-posted at Political Arithmetik.


POLL: Rasmussen Virginia Senate, President


A new Rasmussen Reports automated survey of 500 likely voters in Virginia (conducted 1/3) finds:

  • Former Democratic Gov. Mark Warner leads former Republican Gov. Jim Gilmore (53% to 38%) in a general election match-up for U.S. Senate.

The margin of sampling error is 4.5%


Looking for New Hampshire Exit Polls?

Topics: 2008 , Exit Polls , New Hampshire , The 2008 Race

Looking for leaked exit poll results from New Hampshire? Sorry to disappoint, but whatever their merits, we are unlikely to see any such leaked results until moments before the polls close.

In past years, the network consortium that conducts the exit polls distributed mid-day estimates and tabulations to hundreds of journalists that would inevitably leak. In 2006, however, the networks adopted a new policy that restricted access to a small number of analysts in a "quarantine room" for most of the day and did not release the results to the networks and subscriber news organizations until just before the polls closed (information that did ultimately leak to blogs). As far as I know, that process will remain in place today.

Here are a few tips for making sense of the exit poll data that you do see tonight:

1) An exit poll is just a survey. Like other surveys, it is subject to random sampling error and, as those who follow exit polls now understand, occasional problems with non-response bias. In New Hampshire (in 1992) and Arizona (in 1996)* primary election exit polls overstated support for Patrick Buchanan, probably because his more enthusiastic supporters were more willing to be interviewed (and for those tempted to hit he comment button, yes, I know that some believe those past errors suggest massive vote fraud -- I have written about that subject at great length).

2) The networks rarely "call" an election on exit poll results alone. The decision desk analysts require a very high degree of statistical confidence (at least 99.5%) before they will consider calling a winner (the ordinary "margin of error" on pre-election polls typically uses a 95% confidence level). They will also wait for actual results if the exit poll is very different from pre-election poll trends. So a single-digit margin on an exit poll is almost never sufficient to say that a particular candidate will win.

3) Watch out for "The Prior." At least two networks are likely to post exit poll tabulations shortly after the polls close that will update as the election night wears on (try these links for MSNBC and CNN). Those data are weighted to whatever estimate of the outcome the analysts have greatest confidence in at any moment. By the end of the night, the tabulations will be weighted to the official count. Typically, the exit poll tabulations are weighted to something called the "Composite Estimate," a combination of the exit poll data alone and a "Prior Estimate" that is based largely on pre-election poll results. So if you look to extrapolate from the initial tabulations posted on MSNBC or CNN (as we did here on Election Night 2006), just keep in mind that in the estimate of each candidate's standing in the initial reports will likely mix exit poll and the pre-election poll estimates (not unlike the kind we report here).

Finally, if you would like more information on how exit polls are conducted, you may want to revisit a Mystery Pollster classic: Exit Polls - What You Should Know. Happy New Hampshire Primary Day!

*Clarification: The original version of this post implied that the 1996 overstatement occurred in New Hampshire.

Note: An apology to those who continue to receive the "too many comments" error message when attempting to post a comment here. Suffice it to say, you have not posted too many comments to Pollster.com. We have been trying to squash this bug for months now (without success) and, unfortunately, the very heavy traffic we are experiencing today aggravates the bug.


POLL: Rasmussen New Hampshire Primary


The final Rasmussen Reports automated tracking survey of likely primary voters in New Hampshire (conducted 1/5 through 1/7) finds:

  • Among 1,549 likely Republican primary voters, Sen. John McCain runs at 32%, former Gov. Mitt Romney at 31% in a statewide primary; former Gov. Mike Huckabee trails at 10%, former Mayor Rudy Giuliani and Rep. Ron Paul at 8%.
  • Among 1,774 likely Democratic primary voters, Sen. Barack Obama leads Sen. Hillary Clinton (37% to 30%) in a statewide primary; former Sen. John Edwards trails at 19%, Gov. Bill Richardson at 8%.
  • All other candidates receive less than five percent each. The margin of sampling error is 2% for likely Democratic primary voters and 3% for likely Republican primary voters.


POLL: Suffolk University New Hampshire Primary


The final Suffolk University/7NEWS statewide tracking survey (release) of likely primary voters in New Hampshire (conducted 1/6 through 1/7) finds:

  • Among 500 likely Democratic primary voters, Sen. Barack Obama runs at 39%, Sen. Hillary Clinton at 34%, former Sen. John Edwards at 15%.
  • Among 500 likely Republican primary voters, former Gov. Mitt Romney runs at 30%, Sen. John McCain at 26%, Mike Huckabee at 13%, former Mayor Rudy Giuliani 11%, Rep. Ron Paul 5%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4.4% for each subgroup.


POLL: Reuters/C-SPAN/Zogby New Hampshire Primary


The final Reuters/C-SPAN/Zogby statewide tracking survey of likely primary voters in New Hampshire (conducted 1/5 through 1/7) finds:

  • Among 862 likely Democratic primary voters, Sen. Barack Obama leads Sen. Hillary Clinton (42% to 29%) in a statewide primary; former Sen. John Edwards trails at 17%, Gov. Bill Richardson at 5%.
  • Among 859 likely Republican primary voters, Sen. John McCain leads former Gov. Mitt Romney (36% to 27%) in a statewide primary; former Gov. Mike Huckabee trails at 10%, former Mayor Rudy Giuliani and Rep. Ron Paul both trail at 9%.
  • All other candidates receive less than five percent each. The margin of sampling error is 3.4% for each subgroup.


POLL: ARG NH Primary


The final American Research Group statewide tracking survey of likely primary voters in New Hampshire (conducted 1/6 through 1/7) finds:

  • Among 600 likely Democratic primary voters, Sen. Barack Obama leads Sen. Hillary Clinton (40% to 31%) in a statewide primary; former Sen. John Edwards trails at 20%.
  • Among 600 likely Republican primary voters, Sen. John McCain leads former Gov. Mitt Romney (31% to 24%) in a statewide primary; former Gov. Mike Huckabee trails at 14%, former Mayor Rudy Giuliani at 13%, Rep. Ron Paul at 9%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4% for each subgroup.


Mellman on Word of Mouth in IA/NH

Topics: 2008 , The 2008 Race

In an op-ed in Sunday's New York Times, Democratic pollsters Mark Mellman and Michael Bloomfield presented survey data they collected in Iowa recently showing that "what people say to one another can be as potent as what TV advertisements try to make them think." They found, in particular, that voters who transform themselves "from mere 'talkers' into advocates" were particularly important to the Iowa candidacies of Mike Huckabee and John Edwards:

Whether by chance or design, such citizen advocates created the explosive growth in support for Mike Huckabee and sustained John Edwards, even as both were vastly outspent by their opponents.

Our polling found 23 percent of Republicans were advocates for Mike Huckabee as against 16 percent for Mr. Romney. At 30 percent, Mr. Edwards had the most word-of-mouth advocates among Democrats by a narrow margin, which explains how he was able to remain competitive in Iowa despite his financial disadvantage.

Tonight, a special bonus. Mellman sends along a PDF release with results from two new, just completed surveys of likely Democratic and Republican primary voters in New Hampshire. Some highlights:

Now, in New Hampshire, it is Obama being further buoyed by the greatest net positive talk—23 points more than Clinton and 11 points more than Edwards. Clinton continues to generate significant negative comment. Indeed, almost 70% more Democratic primary voters in New Hampshire are speaking ill of Clinton than of Obama or Edwards. [...]

The only Republican generating truly positive buzz in New Hampshire is John McCain, whose net positive talk score (+38) more than doubles Mitt Romney’s (+15). Rudy Giuliani is at just +6. Interestingly, while Mike Huckabee has generated substantial discussion in New Hampshire as a result of his Iowa victory, much of it is negative, so his score is just +2, with nearly as many GOPers speaking negatively about him as positively.

Mellman informs the survey has no sponsor other than his company.


Luntz: Uses Repeat Participants by Design

Topics: Focus Groups

Earlier today, TPM's Paul Kiel asked Frank Luntz about a participant in his most recent post-debate focus group for Fox News that had also appeared in prior televised focus group:

[W]hen I spoke to Luntz today, he said that he uses repeat participants by design. In a segment to air on Fox News tonight, he said, there should be a "bunch of people" who had been in prior focus groups, some of them participating as early as May of last year. "It allows me to see how people's opinion have changed over time," he explained. "I'm trying to isolate that moment that made the difference."

Before Kiel spoke to Luntz, he called me for a reaction. My own view, quoted in Kiel's article, is that the presence of repeat participants is a troubling indicator about of recruitment quality. But then, I had never heard of this particular application of what Luntz goes on to describe as "anthropology." Draw your own conclusions, I suppose, but the survey researchers I know try to avoid repeat focus group participants.

I wrote a brief introduction to political focus groups, their strengths and weaknesses, back in 2006. Then as now, I also highly recommend this story that aired on NPR's Marketplace in March 2002 on what can go wrong in focus group recruitment (the focus group story begins at about 4:10).

Update - Reader Mike G. leaves this helpful comment:

As someone who worked in market research for four years, I can tell you that it is viable to use the same group. This is what's known as a longitudinal method. However, Frank Luntz should have been up front about this at the beginning of the segment. The entire segment was presented like these voters had not been interviewed previously. This is intellectually dishonest and, in polling, such intellectual dishonesty has no place.


POLL: CNN/WMUR/UNH New Hampshire Primary


A new CNN/WMUR/UNH statewide survey (Results: Dems, Reps) of likely primary voters in New Hampshire (conducted 1/5 through 1/6) finds:

  • Among 599 likely Democratic primary voters, Sen. Barack Obama leads Sen. Hillary Clinton (39% to 30%) in a statewide primary; former Sen. John Edwards trails at 16%, Gov. Bill Richardson at 7%.
  • Among 492 likely Republican primary voters, Sen. John McCain runs at 31%, former Gov. Mitt Romney at 26% in a statewide primary; former Gov. Mike Huckabee trails at 13%, former Mayor Rudy Giuliani and Rep. Ron Paul both at 10%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4% for each subgroup.
  • Note: The previous CNN/WMUR/UNH release was based on interviews from Saturday evening through Sunday afternoon. This current release adds interviews conducted through Sunday evening.


POLL: SurveyUSA South Carolina Primary


A new SurveyUSA automated survey of likely primary voters in South Carolina (conducted 1/4 through 1/6) finds:

  • Among 579 likely Democratic primary voters asked to choose between three candidates, Sen. Barack Obama leads Sen. Hillary Clinton (50% to 30%) in a statewide primary; former Sen. John Edwards trails at 16%. Among African Americans (51% of the sample), Obama leads Clinton 69% to 23%.
  • Among 658 likely Republican primary voters asked to choose between six candidate, former Gov. Mike Huckabee leads former Gov. Mitt Romney (36% to 19%) in a statewide primary; Sen. John McCain trails at 17%, former Sen. Fred Thompson at 11%, former Mayor Rudy Giuliani at 9%, Rep. Ron Paul at 5%.
  • The margin of sampling error is 4.2% for likely Democratic primary voters and 3.9% for likely Republican primary voters.


POLL: Rasmussen SC Dem Primary


A new Rasmussen Reports automated survey of 553 likely Democratic primary voters in South Carolina (conducted 1/6) finds:

  • Sen. Barack Obama leads Sen. Hillary Clinton (42% to 30%) in a statewide primary; former Sen. John Edwards trails at 14%. All other candidates receive less than five percent each.
  • Among African Americans, Obama leads Clinton 58% to 30%. Note, the percent of the sample that is African Americans was not released.
  • The margin of sampling error is 4% for all likely Democratic primary voters.


POLL: USA Today/Gallup National Primary


A new USA Today/Gallup national survey (Gallup results) of 1,023 adults (conducted 1/4 through 1/6) finds:

  • Among 423 Republicans and those who lean Republican, former Gov. Mike Huckabee runs at 25%, former Mayor Rudy Giuliani at 20%, and Sen. John McCain at 19% in a national primary; former Sen. Fred Thompson trails at 12%, former Gov. Mitt Romney at 9%.
  • Among 499 Democrats and those who lean Democratic, Sen. Hillary Clinton and Sen. Barack Obama both run at 33% in a national primary; former Sen. John Edwards trails at 20%.
  • All other candidates receive less than five percent each. The margin of sampling error is 5% for both subgroups.

We will provide links once they are available.


New Hampshire Endgame

Topics: 2008 , New Hampshire , The 2008 Race

1NHEndgame17.png

The New Hampshire endgame polling presents an interesting contrast. The Republican race shows virtually no hint of an "Iowa Bounce." The Democratic race, on the other hand, is showing a huge bounce for Obama and a drop for Clinton. Edwards is largely unaffected.

The charts also show the better performance of the sensitive red-line estimator when things are as dynamic as they have been since Thursday. The Red estimator catches the upturn in Obama support pretty well, while the blue estimator is trying hard to keep up but its "slow to change" nature means it totally misses the timing of the upswing.

If anyone were actually asking if there has been an Obama bounce, surely they are no longer asking.

The Republican side is a bit more sedate, probably because the leader there, John McCain, was hardly a top finisher in Iowa. The upward trend for McCain, and the downward one for Romney, predated the Iowa caucuses. At most the trends we saw earlier have largely continued. Huckabee appears to be the candidate without a bounce, in fact.

These are dynamics we've seen before when Iowa has had an impact. The short interval between Iowa has been much debated. One side says it doesn't allow enough time for an Iowa bounce to be fully felt. I'm of the opposite opinion. The short interval maximizes the effect of Iowa by not allowing time for losers in Iowa to retool their approach and for "added scrutiny" of the Iowa winner to slow their climb. The issue for Clinton and Romney is how to halt what is beginning to look like disastrous slides. With more time between events they would be better able to recover. If New Hampshire is a second loss for both, then both campaigns have to find ways to recover by South Carolina.

Cross-posted at Political Arithmetik.


NH: The View from Monday Afternoon

Topics: 2008 , New Hampshire , The 2008 Race

It's been quite a day for new poll releases. We now have new results from 11 different organizations that continued based on interviews in New Hampshire through Sunday, with two more that wrapped up calling on Saturday. As such, I have updated my table from yesterday's post on the size of the Obama bump.

01-07 NH bump.png

Across all of the polls that interviewed through Sunday night, Obama leads by an average of eight percentage points (37% to 29%), by slightly less (36% to 29%) if we also include the two polls that completed on Saturday. These averages are a near perfect match for our standard and sensitive estimates respectively. In terms of the Obama "bump," his support has gained an average of 8 percentage points; his net gain (the difference in the margins Obama's gain minus Clinton's decline) averages 13 points.

Obviously, Obama's margin has expanded from what we reported yesterday. Add to that the obvious increase since last week and the large number of voters still uncertain about their choice (20% "still trying to make up their minds" on the CNN/WMUR/UNH survey), we should assume that Obama's advantages will likely continue to grow over the next 24 hours.

Charles Franklin should have more soon, including a closer graphic look at the final trends.


POLL: CBS Dem NH Primary


A new CBS News statewide survey (story, results) of 323 likely Democratic primary voters in New Hampshire (conducted 1/5 through 1/6) finds:

  • Sen. Barack Obama runs at 35%, Sen. Hillary Clinton at 28% in a statewide primary; former Sen. John Edwards trails at 19%, Gov. Bill Richardson at 5%.
  • This poll is what pollsters call a "panel-back" survey: "This poll was conducted by telephone January 5-6, 2008 among registered voters in New Hampshire, first interviewed by CBS News and the New York Times November 9-12th, 2007."
  • Senator Clinton "lost almost one in five of her November voters to Obama, and 10% of her voters have gone to Edwards. Obama, meanwhile, has kept 95% of the individual voters he had in November.

All other candidates receive less than five percent each. The margin of sampling error is 5%.


POLL: Fox New Hampshire Primary


A new FOX News/Opinion Dynamics statewide survey (story, results) of likely primary voters in New Hampshire (conducted 1/4 through 1/6) finds:

  • Among 500 likely Republican primary voters, Sen. John McCain runs at 34%, former Gov. Mitt Romney at 27% in a statewide primary; former Gov. Mike Huckabee trails at 11%, former Mayor Rudy Giuliani at 9%, Rep. Ron Paul at 5%.
  • Among 500 likely Democratic primary voters, Sen. Barack Obama runs at 32%, Sen. Hillary Clinton at 28% in a statewide primary; former Sen. John Edwards trails at 18%, Gov. Bill RIchardson at 6%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4% for each subgroup.


POLL: Strategic Vision (R) New Hampshire Primary


A new Strategic Vision (R) statewide survey of likely primary voters in New Hampshire (conducted 1/4 through 1/6) finds:

  • Among 600 likely Democratic primary voters, Sen. Barack Obama leads Sen. Hillary Clinton (38% to 29%) in a statewide primary; former Sen. John Edwards trails at 19%, Gov. Bill Richardson at 7%.
  • Among 600 likely Republican primary voters, Sen. John McCain leads former Gov. Mitt Romney (35% to 27%) in a statewide primary; former Gov. Mike Huckabee trails at 13%, former Mayor Rudy Giuliani at 8%, Rep. Ron Paul at 7%, former Sen. Fred Thompson at 5%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4% for each subgroup.


POLL: Marist New Hampshire Primary


A new Marist College statewide survey of likely primary voters in New Hampshire (conducted 1/5 through 1/6) finds:

  • Among 636 likely Democratic primary voters, Sen. Barack Obama leads Sen. Hillary Clinton (36% to 28%) in a statewide primary; former Sen. John Edwards trails at 22%, Gov. Bill Richardson at 7%.
  • Among 477 likely Republican primary voters, Sen. John McCain runs at 35%, former Gov. Mitt Romney at 31% in a statewide primary; former Gov. Mike Huckabee trails at 13%, Rep. Ron Paul at 8%, former Mayor Rudy Giuliani at 5%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4% for likely Democratic primary voters and 4.5% for likely Republican primary voters.


POLL: Suffolk University New Hampshire Primary


The sixth in a series of Suffolk University/7NEWS statewide tracking surveys of likely primary voters in New Hampshire (conducted 1/5 through 1/6) finds:

  • Among 500 likely Democratic primary voters, Sen. Barack Obama runs at 35%, Sen. Hillary Clinton at 34% in a statewide primar; former Sen. John Edwards trails at 15%.
  • Among 500 likely Republican primary voters, former Gov. Mitt Romney runs at 30%, Sen. John McCain at 27% in a statewide primary; former Mayor Rudy Giuliani trails at 10%, former Gov. Mike Huckabee at 9%, Rep. Ron Paul at 8%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4.4 % for each subgroup.

We will add links as soon as they are available.


POLL: Zogby New Hampshire Primary


The fourth in a series of Reuters/C-SPAN/Zogby statewide tracking surveys of likely primary voters in New Hampshire (conducted 1/4 through 1/6) finds:

  • Among 844 likely Democratic primary voters, Sen. Barack Obama leads Sen. Hillary Clinton (39% to 29%) in a statewide primary; former Sen. John Edwards trails at 19%, Gov. Bill Richardson at 6%.
  • Among 834 likely Republican primary voters, Sen. John McCain runs at 34%, former Gov. Mitt Romney at 29% in a statewide primary; former Gov. Mike Huckabee trails at 10%, former Mayor Rudy Giuliani at 9%, Rep. Ron Paul at 6%.
  • All other candidates receive less than five percent each. The margin of samplingerror is 3.4% for each subgroup.


POLL: ARG New Hampshire Primary


The second in a succession of American Research Group statewide tracking surveys of likely primary voters in New Hampshire (conducted 1/5 through 1.6) finds:

  • Among 600 likely Democratic primary voters, Sen. Barack Obama leads Sen. Hillary Clinton (39% to 28%) in a statewide primary; former Sen. John Edwards trails at 22%.
  • Among 600 likely Republican primary voters, Sen. John McCain leads former Gov. Mitt Romney (35% to 27%) in a statewide primary; former Gov. Mike Huckabee trails at 12%, former Mayor Rudy Giuliani at 10%, Rep. Ron Paul at 7%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4% for each subgroup.


POLL: WBZ/Franklin Pierce NH Primary


A new WBZ/Franklin Pierce statewide survey (story, dems, reps) of likely primary voters in New Hampshire (conducted 1/4 through 1/6) finds:

  • Among 403 likely Democratic primary voters, Sen. Barack Obama runs at 34%, Sen. Hillary Clinton at 31% in a statewide primary; former Sen. John Edwards trails at 20%, Gov. Bill Richardson at 6%.
  • Among 409 likely Republican primary voters, Sen. John McCain leads former Gov. Mitt Romney (38% to 29%) in a statewide primary; former Gov. Mike Huckabee trails at 9%, former Mayor Rudy Giuliani at 8%, Rep. Ron Paul at 7%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4.9% for each subgroup.


POLL: USA Today/Gallup New Hampshire Primary


A new USA Today/Gallup statewide survey (Gallup analysis) of likely primary voters in New Hampshire (conducted 1/4 through 1/6) finds:

  • Among 778 likely Democratic primary voters, Sen. Barack Obama leads Sen. Hillary Clinton (41% to 28%) in a statewide primary; former Sen. John Edwards trails at 19%, Gov. Bill Richardson at 6%.
  • Among 776 likely Republican primary voters, Sen. John McCain runs at 34%, former Gov. Mitt Romney at 30%, former Gov. Mike Huckabee at 13%, former Mayor Rudy Giuliani and Rep. Ron Paul both at 8%.
  • All other candidates receive less than five percent each. The margin of sampling error is 4% for each subgroup.

We will post more links as soon as they are available.


POLL: CNN/WMUR/UNH New Hampshire Primary


A new CNN/WMUR/UNH statewide survey (UNH Dems, Reps) of likely primary voters in New Hampshire (conducted 1/5 through 1/6) finds:

  • Among 341 likely Democratic primary voters, Sen. Barack Obama leads Sen. Hillary Clinton (39% to 29%) in a statewide primary; former Sen. John Edwards trails at 16%, Gov. Bill Richardson at 7%.
  • Among 268 likely Republican primary voters, Sen. John McCain runs at 32%, former Gov. Mitt Romney at 26% in a statewide primary; former Gov. Mike Huckabee trails at 14%, former Mayor Rudy Giuliani at 11%, Rep. Ron Paul at 10%.
  • All other candidates receive less than five percent each. The margin of sampling error is 5% for both subgroups.

We will provide more links as soon as they are available.


NH: View from Sunday Morning

Topics: 2008 , New Hampshire , The 2008 Race

So as of this morning we have seven new polls conducted all or in part after results of the Iowa Caucuses were known. The spin-debate dejour is whether Barack Obama received a "bump," and if so, how much. It is, unfortunately, hard to answer that question given the uncertainty of weekend interviewing and the hard decisions that New Hampshire Democrats are now working to make. Let's look at what we know.

Obama is certainly rising, the only question is by how much. As the table below indicates, all seven polls show some increase in Obama's share of the vote. It ranges between 2 and 10 percentage points, with an average of gain of six points and a median of 5, although the most respected New Hampshire pollster on the list -- CNN/WMUR/University of New Hampshire -- shows a smaller slightly smaller 3-point gain.

01-06%20NH%20summary%28400%29.png

With two notable exceptions (ARG and RasmussenReports), the surveys show a very close race. The average of the seven gives Obama a three-point edge, while the median shows Obama up by two. Our regression trends (which should update on our New Hampshire page shortly) show Clinton with a one-point advantage (33% to 32%) on the standard estimate, but a point down (32% to 33%) on the more sensitive estimate. All of these differences fall well within the the margin of real-world and "sampling" error.

However, Obama appears to be gaining. So how big will the bump be? Firm conclusions are premature for two important reasons.

The first involves the issue of weekend interviewing, or more specifically, surveys based on interviews completed entirely on Friday night and Saturday. Most campaign pollsters are reluctant to put too much faith in interviews conducted at those times, when younger and more mobile voters are less likely to be home. In my 20+ years of looking at surveys conducted for campaigns, I can remember only one we did based solely on Friday and Saturday interviewing. In that case even after we weighted by every demographic variable available to make it comparable to others conducted just days before, we produced a weighted sample that appeared much more engaged in politics and better informed about issues and candidates (and thus, more likely to be "certain" about their initial vote preferences).

On the other hand, I cannot claim much experience with weekend interviewing -- my sample size is just one survey. Media pollsters are obviously more willing to conduct such surveys, particularly over the last weekend before an election. So I am willing to suspend disbelief, although I will have a lot more faith in the releases based on interviews conducted through Sunday night.

An aside: When pollsters like me worry about "weekends" we mean Friday night and Saturday, not Sunday. Actually, late Sunday afternoons and evenings are among the best times to catch people at home, especially in the winter. And I see much less to fear from a survey that begins calling on Friday and finishes on Sunday, so long long as all of the "no answer" numbers from Friday and Saturday get dialed again on Sunday night.

The second and more important reason to be cautious about this Sunday morning snapshot is that New Hampshire voters are still in the midst of a difficult decision. The CNN/WMUR/UNH survey tells us that only 52% of Democrats are "definitely decided" about who they will support, while 26% are "leaning toward someone" and 23% are "still trying to decide." Obama has an advantage over Clinton among the definitely decided (41% to 35%; n=183), while Clinton has a slight edge -- for the moment at least -- among those leaning or uncertain (31% to 23%; n=173).

But as you step away from the trial heat results and look at other internal measures, we see why the choice is difficult. Voters like all three of the leading candidates. For example, among the 82 respondents that are "still trying to decide:"

  • 92% rate Obama favorably, only 3% unfavorably

  • 81% rate John Edwards favorably, only 5% unfavorably

  • 75% rate Clinton favorably, only 5% unfavorably

Those same uncertain voters (n=82) also choose:

  • Obama over Clinton as "most inspiring" (68% to 8%)

  • Clinton over Obama as the candidate with "the right experience" (53% to 4%)

  • Obama over Clinton -- though narrowly and with more uncertainty -- as "most likely to bring needed change" (34% to 22%). Obama has a bigger advantage on this measure (40% to 27%) among those who are "leaning" to their choice.

One more thing. I cannot point to an academic study to prove this, but most campaign pollsters will tell you that when a candidate is gaining, vote preference is usually the last thing to change. The movement usually shows up first on internal measures. So on that score, consider that the UNH survey, which shows the smallest "bump" also shows a huge shift on perceptions of electability. Ten days ago, likely Democratic primary voters in New Hampshire considered Clinton the "candidate with the best chance of defeating the Republican" by a a two-to-one margin (45% to 22%). Obama has closed that margin on the most recent survey to a single percentage point (Clinton 36%, Obama 35%).


 

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR