Articles and Analysis


New Hampshire: What About Monday Night?

Topics: 2008 , New Hampshire , The 2008 Race

Back to the final polls in New Hampshire. One of the statements I have heard from some pundits over the last few days is that pollsters stopped calling on Sunday. While that was true for most of the organizations that conducted surveys over the final weekend, there were four pollsters that continued calling through Monday. Unfortunately, the results they reported do not show a consistent pattern, although the real story may be a bit more complicated.

All four were doing "rolling average" tracking, so their final release used data collected over the preceding two or thee days. If voter preferences changed radically on Monday those changes would only affect one third or half of their data. However, a comparison of their last two releases should give a good indication of whether the Monday night interviews showed Obama's lead expanding or declining, especially since all four showed Obama gaining over the weekend.

As the table below shows, the results are mixed. Two pollsters -- American Research Group (ARG) and Rasmussen Reports -- showed a slightly narrower Obama lead in their final release, but two pollsters -- Suffolk University and Zogby -- showed the Obama lead growing. None of the final shifts were big enough to be statistically significant, so if we take these results at face value, we are left with a picture of essentially random variation.

01-15 monday polls.png

But can we take all of these results at face value?

Consider that on its six releases, the three-day rolling average Zogby tracking reported Obama's support steadily gaining. Their first track (finished just before the Iowa Caucus results were known) showed Clinton leading by six points (32% to 26%). Successive releases had that lead narrowing to four points, then to one point, then showed Obama suddenly leading by 10 points and then by 13 points (42% to 29%).

01-15 Zogby NH.png

The final Zogby release leads with the following sentence:

The big momentum behind Democrat Barack Obama, a senator from Illinois who is seeking his party’s presidential nomination, continued up to the last hours before voters head to the polls to cast ballots in the New Hampshire primary election, a new Reuters/C-SPAN/Zogby daily tracking poll shows.

Yet the next day, in a release titled, "What Happened," John Zogby reports the following:

My polling showed Clinton doing well on the late Sunday night and all day Monday – she was in a 2-point race in that portion of the polling. But since our methods call for a three-day rolling average, we had to legitimately factor the huge Obama numbers on Friday and Saturday – thus his 12 point average lead. Unfortunately, one day or a day–and–a–half does not make a trend and we ran out of time.

So on Tuesday Zogby tells us of Obama momentum that "continued up to the last hours." On Wednesday he says the momentum ran out on Sunday afternoon. Some would see a contradiction there. Rather than focusing on the verbiage, let's focus on the numbers. Perhaps my mathematically inclined readers can come up with a realistic set of hypothetical single day results (and half day results for early and late Sunday) that can reconcile Mr. Zogby's data reported on Monday and Tuesday with his comments on Wednesday. I cannot.

I know, from personal experience, that Mr. Zogby gets very angry about suggestions that the remarkable last minute surge he was willing to report on the eve of the 2004 New Hampshire primary ("For Kerry the dam burst after 5PM on Monday") represents a last minute "correction" intended to bring his results into line with those of other pollsters.

While I assume he will be similarly unhappy about this piece, he has an easy remedy: Release the one-day results (and part-day Sunday results) used to calculate his rolling-averages. Each daily sample should exceed 250 interviews (more than some complete surveys we have reported in recent months). If nothing else, the day-by-day results will further our understanding of what happened in New Hampshire. And if the single-day results are consistent with both the previously reported data and Mr. Zogby's post election claims, I will happily apologize for any implication to the contrary.

On a related note, the Suffolk University survey, the one that also showed Obama's lead continuing to expand in interviews conducted on Sunday and Monday night, provided full cross-tabulations for all of their data releases (these are still online - see the links in the left column of the Suffolk web site). While the Suffolk University pollsters do not break out single-day results, they do provide the demographic and regional composition of each days' sample. Their sample composition in terms of age, gender and region showed only trivial variation over the last 3 to 4 releases -- certainly nothing that would explain away the continuing improvement in the Obama margin in their final tabulations. For what it's worth, the Suffolk poll featured the largest "undecided" percentage and the smallest sample sizes of the four pollsters that continued to call on Monday.

Finally, the Rasmussen Reports result is also intriguing, because their final release added 571 interviews conducted on Monday to to 1,203 conducted on Saturday and Sunday. As such, we can do a rough extrapolation, which shows Obama leading by only a point (35% to 34%, but there is much room for rounding error here) in the Rasmussen interviews conducted Monday night. Rasmussen hinted at this result in his own post-mortem but did not release single night numbers. For the Rasmussen data, at least, the numbers add up.

PS: Robert Wright, in an election day email that Mickey Kaus blogged earlier toay, noticed a similar pattern in the final CNN/WMUR/UNH release that added 258 interviews of likely Democratic primary voters conducted Sunday night to the 341 gathered on Saturday and early Sunday that they had released previously. The Obama margin narrowed by a single percentage point, neither statistically significant difference nor enough to enable a meaningful extrapolation, though it is consistent with both the direction of the final Rasmussen and ARG releases.


Mark Lindeman:

If we assume that Zogby's numbers for each day 12/31-1/3 were equal; if we assume that every day is weighted equally in each rolling average; and if we discount rounding error -- then we would have Obama +2 on Friday, +6 on Saturday, +22 on Sunday, and +11 on Monday.

None of those assumptions is safe; it's worth doing the exercise in order to realize that at least the Monday margin was probably less than the Sunday margin. (This is less obvious when looking at the averages, which show Obama continuing to pull away -- because the Monday number replaces the smaller Friday number.) But it's really hard to figure how Zogby had "huge Obama numbers" on Friday and Saturday. If Obama's margin on Monday isn't larger than the Friday margin, then why does the final average go up?

I can't forget that Zogby called the 2004 general election for Kerry (311-213) at 5:00 pm on election day -- and yet he boasts, "No public pollster can match Zogby's distinguished track record of correctly polling the last 3 Presidential races." Oh-kay.



Interesting swing in votes. If this were in Chicago, the answer would be obvious.



Mr. Lindeman understates Zogby's 2004 problem. As I recall it -- I may have the details wrong -- Zogby released an election day poll midday in 2004 showing the race an electoral college dead heat with the winner to be determined by the too-close-to-call states of Pennsylvania (!) and Virginia (!!). Then late in the day, after the now known to be bunk exit polls started filtering in, Zogby rushed in with a new "final" poll showing an electoral college landslide for Kerry and yet, almost unbelievably, a popular vote win for Bush.
It does not take a cynic to see that Zogby tried to have it every conceivable way. And when it looked like Kerry might be winning, Zogby suddenly discovered a huge electoral vote tilt for Kerry even though his polling showed Bush ahead in the popular vote.
His results in 2006 and in New Hampshire this year are more of the same. He has these great ex post arguments about how he was really right all along, but if you relied on what he said before the votes were counted, you would be generally wrong.


Moral Hazard:

What Zogby meant to say was "No public pollster can match Zogby's DISINGENUOUS track record of correctly polling the last 3 Presidential races."



There is an interesting analysis in a dKos diary that I'd be curious to see critiqued. One of the key points is that primaries are hard to poll, not just because of more volatile turnout, but because the number of true undecideds is very high, right up to the end (at least, in highly contested primaries).

To illustrate the point, he (or she) uses the NH exit poll question that asks (paraphrasing) "at what point did you make up your mind who to vote for?" That data point is then factored in to late polling.

Days before election -- % of voter who had really decided
30: 33%
14: 50%
7: 60%
3: 75%
1: 85%

That means that on the weekend before, despite what the polls were reporting, a quarter of the voter had not really decided. In a general election, this is about the same percentage as had made up their minds a month before the election. So the two really are VERY different. I'd be curious to see if, before pushing leaners, what the polls were actually seeing with respect to undecideds.


Post a comment

Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.