Mark Blumenthal | January 15, 2008
Topics: 2008 , New Hampshire , The 2008 Race
Back to the final polls in New Hampshire. One of the statements I have heard from some pundits over the last few days is that pollsters stopped calling on Sunday. While that was true for most of the organizations that conducted surveys over the final weekend, there were four pollsters that continued calling through Monday. Unfortunately, the results they reported do not show a consistent pattern, although the real story may be a bit more complicated.
All four were doing "rolling average" tracking, so their final release used data collected over the preceding two or thee days. If voter preferences changed radically on Monday those changes would only affect one third or half of their data. However, a comparison of their last two releases should give a good indication of whether the Monday night interviews showed Obama's lead expanding or declining, especially since all four showed Obama gaining over the weekend.
As the table below shows, the results are mixed. Two pollsters -- American Research Group (ARG) and Rasmussen Reports -- showed a slightly narrower Obama lead in their final release, but two pollsters -- Suffolk University and Zogby -- showed the Obama lead growing. None of the final shifts were big enough to be statistically significant, so if we take these results at face value, we are left with a picture of essentially random variation.
But can we take all of these results at face value?
Consider that on its six releases, the three-day rolling average Zogby tracking reported Obama's support steadily gaining. Their first track (finished just before the Iowa Caucus results were known) showed Clinton leading by six points (32% to 26%). Successive releases had that lead narrowing to four points, then to one point, then showed Obama suddenly leading by 10 points and then by 13 points (42% to 29%).
The final Zogby release leads with the following sentence:
The big momentum behind Democrat Barack Obama, a senator from Illinois who is seeking his party’s presidential nomination, continued up to the last hours before voters head to the polls to cast ballots in the New Hampshire primary election, a new Reuters/C-SPAN/Zogby daily tracking poll shows.
Yet the next day, in a release titled, "What Happened," John Zogby reports the following:
My polling showed Clinton doing well on the late Sunday night and all day Monday – she was in a 2-point race in that portion of the polling. But since our methods call for a three-day rolling average, we had to legitimately factor the huge Obama numbers on Friday and Saturday – thus his 12 point average lead. Unfortunately, one day or a day–and–a–half does not make a trend and we ran out of time.
So on Tuesday Zogby tells us of Obama momentum that "continued up to the last hours." On Wednesday he says the momentum ran out on Sunday afternoon. Some would see a contradiction there. Rather than focusing on the verbiage, let's focus on the numbers. Perhaps my mathematically inclined readers can come up with a realistic set of hypothetical single day results (and half day results for early and late Sunday) that can reconcile Mr. Zogby's data reported on Monday and Tuesday with his comments on Wednesday. I cannot.
I know, from personal experience, that Mr. Zogby gets very angry about suggestions that the remarkable last minute surge he was willing to report on the eve of the 2004 New Hampshire primary ("For Kerry the dam burst after 5PM on Monday") represents a last minute "correction" intended to bring his results into line with those of other pollsters.
While I assume he will be similarly unhappy about this piece, he has an easy remedy: Release the one-day results (and part-day Sunday results) used to calculate his rolling-averages. Each daily sample should exceed 250 interviews (more than some complete surveys we have reported in recent months). If nothing else, the day-by-day results will further our understanding of what happened in New Hampshire. And if the single-day results are consistent with both the previously reported data and Mr. Zogby's post election claims, I will happily apologize for any implication to the contrary.
On a related note, the Suffolk University survey, the one that also showed Obama's lead continuing to expand in interviews conducted on Sunday and Monday night, provided full cross-tabulations for all of their data releases (these are still online - see the links in the left column of the Suffolk web site). While the Suffolk University pollsters do not break out single-day results, they do provide the demographic and regional composition of each days' sample. Their sample composition in terms of age, gender and region showed only trivial variation over the last 3 to 4 releases -- certainly nothing that would explain away the continuing improvement in the Obama margin in their final tabulations. For what it's worth, the Suffolk poll featured the largest "undecided" percentage and the smallest sample sizes of the four pollsters that continued to call on Monday.
Finally, the Rasmussen Reports result is also intriguing, because their final release added 571 interviews conducted on Monday to to 1,203 conducted on Saturday and Sunday. As such, we can do a rough extrapolation, which shows Obama leading by only a point (35% to 34%, but there is much room for rounding error here) in the Rasmussen interviews conducted Monday night. Rasmussen hinted at this result in his own post-mortem but did not release single night numbers. For the Rasmussen data, at least, the numbers add up.
PS: Robert Wright, in an election day email that Mickey Kaus blogged earlier toay, noticed a similar pattern in the final CNN/WMUR/UNH release that added 258 interviews of likely Democratic primary voters conducted Sunday night to the 341 gathered on Saturday and early Sunday that they had released previously. The Obama margin narrowed by a single percentage point, neither statistically significant difference nor enough to enable a meaningful extrapolation, though it is consistent with both the direction of the final Rasmussen and ARG releases.