Mark Blumenthal | October 5, 2007
Topics: 2008 , The 2008 Race
Earlier this morning, I posted a response to a comment on yesterday's entry from reader Danielle Clark that on reflection is worth discussing further on the main page. Her comments concern the ways that pollsters handle "undecided" voters:
For those wondering why Obama's poll numbers seem surprisingly low, check the raw results as opposed to the press releases. If the raw results aren't posted on the polling company's site, there's probably a reason for that. ;-)
Awhile back I learned it's considered acceptable to drop, re-allocate or report "Don't Know" answers - it all depends on the polling company's choice, and they don't tend to disclose which they did. This can cause misleading results, especially at this point when there are still a lot of people who don't know enough about Obama (and other candidates) to want to commit to one.
For example, a poll may find 5% support Obama, 10% Clinton, 10% others, and 75% don't know. It's considered acceptable to report this as 20% support Obama, 40% Clinton, and 40% others - which implies everyone asked had an opinion of some sort.
Another reason to insist on seeing the raw data is to know what questions were asked before the question one's interested in - those can clearly shape the answer.
She also linked to a useful paper on "Interpreting Polls " put out by a parliamentary librarian in Australia. Aside from our regular commentary here, I would also recommend the resources available on the new web page of the American Association for Public Opinion Research (AAPOR).
Danielle was right to recommend examining the verbatim text of survey questions and the way pollsters calculate their results. Where available, we include links to the most complete report of results in our "poll update" posts and in the tables that appear below the charts in our Polls section. The Polling Report also provides verbatim text for poll questions. Follow those links to see the full results and verbatim text when available.
However, we need to be careful to distinguish between tables or references in news stories that omits the results for the undecided category and those that reallocate undecided voters as Danielle describes.
In the US, virtually all polls are now reporting trial heat results based on calculations that include the undecided or "don't know" category. Some pollsters will put out "projections" based on their final poll before the election that use various methods to reallocate undecided voters (the Polling Report has a handy set of tables showing both "vote projections" and "final trial heats" from 2004 side by side). But I know of no pollsters putting out any such projections right now.
One way to check, if the undecided category is omitted, is to add up the results to each candidate. If they total 100%, then the results have almost certainly been recalculated.
More to the point, all of the results posted and plotted in the charts and tables in the Polls section of Pollster.com -- including the numbers discussed in yesterday's post -- are based on calculations that include the undecided category.
It is also important to distinguish between the undecided category on a trial heat question and separate questions that probe how certain or "decided" voters say they are. The undecided category tends to understate the real potential for change in a political contest. In recent weeks have written about questions asked on recent LA Times/Bloomberg and CNN/WMUR/UNH polls showing huge number of voters in Iowa and New Hampshire that are still less than certain about their choice.
Danielle may be thinking about computations that take those "uncertain" voters into account that Obama himself has been touting on the campaign trail. For example, NBC's First Read reported the following recent comments by Barack Obama at a rally in New Hampshire:
[Obama] also referred to a recent poll that had Sen. Clinton in the lead by twenty points, and said that it meant nothing besides what percentage of people were supporting which candidate. "Twenty percent of voters are with Sen. Clinton, 10 percent are with me, Edwards has about 5 percent and the rest of the candidates have less than that."
Citing that same poll, Obama said the most important fact overlooked by the pundits were the number of undecideds still up for grabs. "There are 55 percent of voters who are undecided. That's 55 to 60 percent of the people who are waiting to hear from you," he told the crowd of campaign volunteers made up of students from colleges in Massachusetts and across New Hampshire as well as local volunteers.
Obama was probably referring to the recent CNN/WMUR/UNH poll that gave Clinton a 22 point lead over Obama (with only 9% "undecided"), but also showed 55% of New Hampshire likely Democratic primary voters saying on a separate question that they are "still trying to decide" which candidate to support. If I set aside the uncertain 55% and recalculate vote preference numbers among those who say they are "definitely decided" or "leaning," I get 23% for Clinton, 10% for Obama and 6% for Edwards.