Articles and Analysis


The Problems of Primary Polling

Topics: 2008 , Jay Cost

I have written a lot this year about the surprising level of indecision about the presidential race among many Democrats on the eve of the primary elections, and the problems that uncertainty creates for polling. Jay Cost weighs in this week with an essay that approaches the same issue from a slightly different perspective. His take is especially relevant now that the primaries have moved beyond hotly contested states like Iowa and New Hampshire to states where information levels about the candidates are significantly lower.

Cost reminds us that "average voters do not pay much attention to politics" and that in general elections, partisan identification serves as the "cognitive heuristic" or "mental shortcut" that facilitates decision making despite low information. He points out that party identification is an "incredibly precise predictor of vote choice" and that it makes for stability in poll measurements:

Accordingly, we will see the polls vary only a little bit throughout the campaign. Oftentimes, they will break in late October or even early November. However, the magnitude of the break will be relatively modest.

However, as Cost points out, primary elections are different:

In a primary campaign, voters must choose among candidates who are all of the same party. Partisanship therefore does not enter into their decisions. It is a non-factor. I think this might be inducing the wild swings in the polls. The polls are varying because the voters are; the voters are varying because their partisanship is not stabilizing their preferences. [...]

It thus should be unsurprising that candidate personalities are so influential in voters' decision-making processes. How else do you make determinations when party distinctions are non-existent? Candidates often try to create clear contrasts, but these usually amount to making mountains out of molehills. The average voter is not really paying much attention, anyway. Thus, they have to go by their personal evaluations of the candidates.

And when those personal evaluations are mostly positive, some voters are having a hard time making up their minds. So their choices, as described to pollsters, may be tenuous. Cost's essay is good; go read it all.


Chuck Miller:

The reason the pollsters are having a huge problem (especially on the Dem side) is primarily with their model of "Likely Voters". With all the new voters in the mix, turnout is much higher and the their estimates of support off considerably.

The new voters simply don't get polled.

We have to go back no further that last week's Virginia primary where the average Obama number the week before was 54%... 10% off his final number of 64%.

If you recall Iowa, the Iowa Poll was criticised for being too high for Obama because the turnout and new voter estimates were "so off"... but it was right on the number.

The pollsters are wrong because their models are not well-suited to this series of primaries.


Daniel T:

Chuck. I agree with you! I have been harping on this since NH: the problem is sampling error. The screens simply are not working and the population polled does not represent the population voting.

As for Jay's post, what a bunch of nonsense.

First, the idea that only 10% if Americans are independent is pure nonsense. The only reason many independents "lean" one way or the other is because pollsters ask them that question. Of course there are very few "true neutral" people in the world. That doesn't mean that a person who is asked by a pollster to lean on way or the other socially or psychologically identifies with that party; they are simply responding to the pollster query. In fact, all political science data of the last 50 years shows the *decreasing* importance of party affiliation. Jay is just wrong on this point as a factual matter.

Second, Jay writes "Uncertainty is the consequence of inattention." This, again, is simply untrue. I don't know where he took a psychology class but it must have been on Mars or some place. I cannot think of a major psychological theory, or even a minor one, that would agree with this statement. In fact, all major psychological theories would hold that uncertainty is the result of *too much* information, i.e. information overload. The essence Jungian psychology, for example, is that personalities and opinions develop as a response to the uncertainties we experience as we sort through the vast amount of information coming into our brains. But more importantly, major psychological theories would hold that the amount of information, by itself, does not create uncertainty; rather, uncertainty results from an inability to *process* information effectively. To use the precise terms, uncertainty is not a failure of psychological aprehension; it's an issue of psychological procession. Since "inattention," by defination relates to aprehension or perception, it cannot be this that *causes* uncertainty.

Jay's post illustrates the problem with "social" commentators who have no background in the fields they are commenting on.


Daniel T:

Let me add that Mark's response to Jay's piece is nonresponsive. We already know that "some voters are having a hard time making up their minds" because, uh, that's what indecision means. The question of why voters are having a hard time making up their minds is a seperate question from why polls are wrong. Polling cannot measure what isn't there and if a voter is truly indecisive the only thing a poll can do is reflect that.

One thing that has started to bug me is that their seems to be an underlying notion that the point of polling is to predict the future. This is simply wrong. All polling can do is measure what the state of the voter's mind is at the moment he or she is polled. If polling represents the actual state of the electorate at the time of the poll, it has succeeded; there is nothing more that it can do. The issue in NH and other places is simply this: was the actual vote a result of a change in events between the poll and the vote or was it an error in sampling. In neither case should a pollster care why a voter behaved as he or she did. if the difference was the result of an actual change in events, then there is nothing that the pollster could have done about that and it unfair to claim that the polls were wrong. On the other hand, if the error was in the sampling, then the question is not only: why did people support Hillary. The question is also why did the people who supported Hillary actually vote. A person's motivation for voting are entirely different than a person's motive to support a person. Forming a positive opinion about a person is not the same thing as taking concrete action to vote for that person. So, despite all it's problems, even if the conclusion of Jay's essay is true (it's about personalities)we still haven't gotten anywhere. We only have explained why people supported a particular candidiate, not why they took the additional step of voting for that person. As all the exit polls show, just because you agree with a person does not mean you'll vote for that person; electibility plays a role. Taking it one step further, just because you support a person doesn't mean you'll go to the polls at all.

From this perspective, NH could be explained not by an error in polling but by the fact that there was a late breaking move to Hillary (the crying incident) AND she had an organization in place to take advantage of that fact. Jay's essay implictly assumes that support=vote and that is simply not true. In this sense, it could be that the Zogby poll in CA actually had the pro-Obama factor excactly right from a polling perspective. But because of Obama's late breaking surge, he did not have the political organization in place to get out the vote and thus lost. Zogby pull would then not be "wrong"; it would be perfectly accurate. It would be wrong in the sense that it measured the wrong thing, and that is purely an issue of sampling related to "likely voters".



I believe the Zogby poll in reference to California was accurate. However, it was due to all the people who had already voted that shifted the tide in Hillary's favor. There were too many absentee ballots and early voters for Obama's momentum to overcome. Still incredibly, that was some amazing shift in California. I think that is reflected in the national polls, where he is now leading. This is not to say that all those who voted for Hillary have changed their mind, but rather that Obama has engaged so many new people to his side.


Nick Panagakis:

I agree that there may have been a problem with likely voter screens, particularly past voting. Record turnout could have been anticipated based on the early states. In Illinois, I cut the past voting question from the likely voting screen. Another factor was the calendar. Many states moved their primary up to dates much earlier than prior years when the nominations had already been decided by the time they got to vote.

I disagree that information level is necessarily lower in later states. There are fewer candidates in later states compared to the clutter at the outset. Information level should be inversely related to the number of choices.



Post a comment

Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.