Pollster.com

Articles and Analysis

 

Are Primary Polls Meaningful?

Topics: 2008 , The 2008 Race

Don't get me wrong. I love polls. Survey research provides us the most accurate and realistic view available of the current attitudes and preferences of the American electorate. Yet, having said that, I find myself frequently frustrated with the tone of much of the poll analysis I hear and read in the news media. Some of the coverage focuses on the national primary polls as if we are on the eve of a national primary.** Others analyze each successive poll - national or statewide -- as if all of the voters were sitting in a jury box, following every campaign development with rapt attention and regularly adjusting their preferences accordingly.

After more than 20 years of conducting polls for political campaigns, I can assure you: It just doesn't work that way.

So it's probably not surprising that I find much to recommend in a Los Angeles Times op-ed in yesterday's Los Angeles Times (rather, from July 2007 but discovered by me yesterday*** via Dickerson) written by two campaign consultants, Democratic pollster Mark Mellman and Republican media consultant Mike Murphy (neither is working for a presidential candidate in 2008, although Mellman polled for Kerry in 2004 and Murphy worked for McCain in 2000). They see strains of the "Heisenberg principle" in what they describe as the "absurdly early start" of Campaign 2008:

It is reminiscent of the Heisenberg uncertainty principle we all heard about in high school physics class. Professor Werner Heisenberg postulated that "the more precisely the position is determined, the less precisely the momentum is known." Applied to the presidential race, this suggests that the more we measure how the candidates stand now, the less we may know about where things are going to end up - because the measurement itself can render the findings inaccurate.

The noisy onslaught of public opinion polling in the media so early in the process would amuse the good professor, because the numbers are really little more than a vain attempt to measure something that hasn't happened. Although the political and media elites may think the campaign is in full swing, with the fortunes of each candidate rising and falling with every new poll, the truth is that voters - the ones who are really going to decide this race - don't start the campaign until much later.

Because voters are not required to make a decision until election day, they remain open at this stage in the race to new information, alternative perspectives and late-breaking developments - all of which render today's poll results, to one degree or another, meaningless.

I'm not sure I agree with "meaningless." That word applies if you look to the current results of the trial heat questions that we chart here at Pollster as conclude that they represent final, informed decisions rather than potentially momentary preferences. However, follow the links and dig deeper and these same surveys hold a lot of meaningful data on how much voters know, and why they prefer one candidate over another, if only for the moment.

Take, for example, the point that Murphy and Mellman make above, that voters in the key early states reaming open to new information and late developments. Their argument receives strong support in the most recent surveys in Iowa, New Hampshire and South Carolina by the L.A. Times. Start with the Democrats. As the table below shows, while more than 80% have a current vote preference (i.e. only 13% to 17% are "undecided"), 59% of the likely caucus goers in Iowa and nearly half in New Hampshire (45%) and South Carolina (47%) say they still "might end up voting for someone else."

09-19%20lat%20certainty%20dems.png

The difference is even greater among Republicans. Roughly the same number (83% to 87% depending on the state) have a vote preference, but 50% to 70% say they "might end up voting for someone else" on election day.

09-19%20lat%20certainty%20gop.png

Voters current preferences have meaning, but they are subject to change, even in the early states. And as history tells us, the results in Iowa and New Hampshire can have a profound impact on voter preferences nationally. So read all of the Murphy-Mellman piece, and stay tuned...

**An update: Speaking of the impact of early primaries, Kathy Frankovic of CBS News devoted her weekly column to the question of "whether national polls about the nomination mean anything now:"

In a nationwide poll that CBS News and The New York Times conducted just before the 1984 New Hampshire Democratic primary, Walter Mondale held what CBS News and The Times characterized as the largest lead ever seen in national polls in the race for a nomination. Fifty-seven percent of Democratic primary voters in that poll chose Mondale, and only 7 percent chose Gary Hart. However, the day after that poll was reported, Gary Hart beat Mondale handily in the New Hampshire primary. Mondale did go on to win the nomination (and lose the election), but the timing of our poll report, and its discrepancy with the New Hampshire outcome raised many questions. Should we even have conducted a national poll before a primary that would mean more to the nomination process than any national poll?

The lesson of 1984 is not to put too much trust in national polls as predictors of primary outcomes. Is there anything that we CAN learn about voters from national polls now? We can certainly see if there are differences between the general population and the people who say they will vote in the primaries or caucuses. We can also gather clues about the possible composition of the primary electorate. Do the most likely voters seem committed to certain candidates? Do they have different issue preferences?

She continues with answers to those questions gleaned frome recent CBS/New York Times polls. Read it all.

***Oops. I obviously misread the date. I think what was true in July remains true today, though Mellman and Murphy might disagree. Thanks to Art for the edit

 

Comments
Art:

Great post Marc, but the opinion piece by Messrs Mellman & Murphy has a July 5, 2007 date, not yesterday as your post claims.

____________________

georgep:

The "might vote for someone else" number has to be qualified furter, since we have more data available.


Taking the example of Iowa:

From the LA/Bloomberg poll 9/10

Iowa

Clinton Edwards Obama
Certain to vote for: 54 41 30
May change later: 46 59 69

So, the qualifier here is that Obama's "may change later" number is extremely high at 69%. That skews the averages to above 50% (to 59% in the article you cite) when indeed just taking Clinton's numbers by themselves gives a different picture. Namely that a majority of her current supporters intend to stay with her - 54%, and another 46% may be open to change their minds. While 46% is still a substantial number of possible changers, it is important to realize that the "might change" numbers differ dramatically from candidate to candidate, which makes the 59% "might change" number a bit hard to qualify, as Obama's very soft support flows into that number. This article makes it appear as if almost 60% are open to change their minds (which is technically true,) but does not mention that it differs greatly, depending on which candidate you are looking at, and that that particular metric on the Democratic side, once broken down, provides us yet another flashpoint that the nomination has actually moved stronger towards Clinton, as her particular "might change mind" numbers are much lower than Edwards and Obama's.

____________________

True, although one could also postulate that this is evidence Obama inherantly appeals more to the uncertain voter than the other two candidates, giving him the most room to grow with the largest group of voters in the key primary states.

____________________

georgep:

Shadow, I am not really disagreeing with the "hidden vote" potential, but the crux of this post here is based on an actual poll finding (not hidden vote potential) and needed some clarification, since broken down into the three principals involved in the Democratic primaries we have a different picture emerging from the one claimed. Namely, that Clinton's current support is "decided" whereas a smaller contingent of her support "might still change their minds." The 59% "might change mind" number is skewed dramatically by Obama's very high "might change mind" number of 69%.

____________________

SF Bay:

georgep, nice analysis or the analysis. Just goes to show, you have to look at all of the data to really get an accurate picture of the polling information. Thanks for taking the time.

____________________

Mark Blumenthal:

For those interested, the numbers that georgep cites are here

Georgep: For what it's worth, "the crux of my point" was not about Hillary Clinton. However, I'm not sure how the data you cite supports a "different picture" in which "Clinton's support is 'decided.'"

The results, taken at face value, indicate that 54% of Clinton's support in Iowa is "decided." That is better than the 41% for all candidates, true. But if you think that Clinton can prevail in Iowa if she holds only 15% of the vote there (54% of the the 28% that currently support her, according to the LA Times), you haven't followed past caucus results very closely.

Of course, given these results, it is also possible that Obama or Edwards could experience a meltdown while Clinton (or another candidate) rises dramatically. And that was the "crux of my point."

____________________

Scott Pauls:

So, how does this compare to 2004 Iowa polls? Specifically, are there iowa polls at a similar distance out with "change your mind" numbers we can compare to since we know the iowa outcome in 2004?

I'm in NH and of the 100 or so politically active Dems I know, over half would claim they have a leaning but could change their mind. Knowing them, however, I suspect they won't unless something large happens. As it likely won't I consider my own "personal poll" rather fixed.

____________________

Scott Pauls makes a key point here - while people aren't sure how they are voting next year, they do have opinions and something will have to happen to change those. The only movement over the last couple of months has been toward Clinton. More Dems also think she will be the nominee and her personal approval rate within the party has improved.
I read the Murphy-Mellman piece when it came out and thought it was like a General fighting the last war. This election cycle is unlike any previous one due to the extreme front-loading. There may be less time to change voter opinions after Iowa and very little time to get focus with advertising after the holiday season.No one knows the exact effect of this, but I would be very nervous about these numbers if I was Obama or Edwards and their more aggressive attitude to Clinton of late indicates that they are indeed nervous.

____________________

georgep:

Mark, I agree that the point has validity when seen as a whole. Of course, if Clinton loses all or a good portion of the support she has that is currently claiming to still "might change their minds," then she loses Iowa, no question. But when you make the distinction and go deeper into the numbers, then a picture emerges that, given that Clinton's support is much more "decided" than Obama's (to take those two as an example) there appears to be a higher chance that Obama loses more of his current support (as 69% of it is currently "soft") and that things move even more into Clinton's direction. I see the overall point that none of the candidates can afford to do without a good portion of their current "soft" supporters, if they lost all of them they would lose. However, I think a strong case can be made that when you parse the "soft" support numbers down to the individual candidates that Clinton is in good shape as already almost 60% of her support is said to be "committed to her" whereas in the case of Obama you have to deduct that his chances (even though all 3 candidates appear close together in Iowa polls) are diminished, because most of his support appears very soft (to the tune of a whopping 69%.) In other words, Obama can't afford to lose ANY of those 69% who are "soft" supporters, which seems a very hard undertaking, given the size of his "soft" support number. Clinton can't afford to lose many of her 41% "soft" support, Edwards can't lose many of his 51% "soft" supporters. The probabilities are on Clinton's side here (Obama stands to lose more supporters, as almost 70% claim to be somewhat soft as it is.) All I was pointing out is that the "might change mind" numbers given in the article cited were for all 3 candidates, which deserved some parsing and further analysis because of the tremendous differences shown when you do parse the numbers down further (due to Obama's poor support numbers in the LA/Bloomberg poll.)

However, I do accept a counterargument that Clinton, being the almost quasi-incumbent in this race, has a lesser chance to in the end get a lot of the truly undecideds going for her, as they probably would have already made that decision for her in the first place, although the caveat here is that she is not a true incumbent (like a sitting president running) or a true quasi-incumbent (like a sitting VP running for the nomination,) so we have very little data to go by in a case like this.

____________________

georgep:

Mark, I agree that the point has some validity when seen as a whole. Of course, if Clinton loses all or a good portion of the support she has that is currently claiming to still "might change their minds," then she loses Iowa, no question. But when you make the distinction and go deeper into the numbers, then a picture emerges that, given that Clinton's support is much more "decided" than Obama's (to take those two as an example) there appears to be a higher chance that Obama loses more of his current support (as 69% of it is currently "soft") and that things move even more into Clinton's direction. I see the overall point that none of the candidates can afford to do without a good portion of their current "soft" supporters, if they lost all of them they would lose. However, I think a strong case can be made that when you parse the "soft" support numbers down to the individual candidates that Clinton is in good shape as already almost 60% of her support is said to be "committed to her" whereas in the case of Obama you have to deduct that his chances (even though all 3 candidates appear close together in Iowa polls) are diminished, because most of his support appears very soft (to the tune of a whopping 69%.) In other words, Obama can't afford to lose ANY of those 69% who are "soft" supporters, which seems a very hard undertaking, given the size of his "soft" support number. Clinton can't afford to lose many of her 41% "soft" support, Edwards can't lose many of his 51% "soft" supporters. The probabilities are on Clinton's side here (Obama stands to lose more supporters, as almost 70% claim to be somewhat soft as it is.) All I was pointing out is that the "might change mind" numbers given in the article cited were for all 3 candidates, which deserved some parsing and further analysis because of the tremendous differences shown when you do parse the numbers down further (due to Obama's poor support numbers in the LA/Bloomberg poll.)

However, I do accept a counterargument that Clinton, being the almost quasi-incumbent in this race, has a lesser chance to in the end get a lot of the truly undecideds going for her, as they probably would have already made that decision for her in the first place, although the caveat here is that she is not a true incumbent (like a sitting president running) or a true quasi-incumbent (like a sitting VP running for the nomination,) so we have very little data to go by in a case like this.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR