Pollster.com

October 5, 2007

Tribe Time Remainders*

By Mark Blumenthal

Kathy Frankovic revisits the classic "forbid or allow" question experiment in the context of the Mahmoud Ahmadinejad speech at Columbia University.

David Hill sees hope for conservatives in a recent Gallup poll result on the Supreme Court.

Adam Nagourney reviews the data on New Hampshire's independent voters.

Frank Newport says the national polls should "cause significant concern" for the Romney and Obama campaigns.

Marjorie Connelly (polling director at the New York Times) examines how much voters "consider the candidates' personal lives" as well as their records.

Matt Stearns quotes "polling experts" who say Hillary Clinton's negatives "may not be politically fatal or even much of a drag."

Greg Sargent has yet another polling and strategy memo from the Giuliani campaign.

Doug Schoen argues that Edwards is the most electable (via Ben Smith) in a column that may surprise Glenn Greenwald.

Carl Bialik examines the way web sites featuring user ratings of "rate beer, restaurants, board games and movies" compile their rankings (for fans of Thomas Bayes).

*Because you can take the boy out of Cleveland...

By Mark Blumenthal on October 5, 2007 9:25 PM | Permalink | Comments (1) | TrackBacks (0)

Yet More on Undecided vs. "Decided"

By Mark Blumenthal

Earlier this morning, I posted a response to a comment on yesterday's entry from reader Danielle Clark that on reflection is worth discussing further on the main page. Her comments concern the ways that pollsters handle "undecided" voters:

For those wondering why Obama's poll numbers seem surprisingly low, check the raw results as opposed to the press releases. If the raw results aren't posted on the polling company's site, there's probably a reason for that. ;-)

Awhile back I learned it's considered acceptable to drop, re-allocate or report "Don't Know" answers - it all depends on the polling company's choice, and they don't tend to disclose which they did. This can cause misleading results, especially at this point when there are still a lot of people who don't know enough about Obama (and other candidates) to want to commit to one.

For example, a poll may find 5% support Obama, 10% Clinton, 10% others, and 75% don't know. It's considered acceptable to report this as 20% support Obama, 40% Clinton, and 40% others - which implies everyone asked had an opinion of some sort.

Another reason to insist on seeing the raw data is to know what questions were asked before the question one's interested in - those can clearly shape the answer.

She also linked to a useful paper on "Interpreting Polls " put out by a parliamentary librarian in Australia. Aside from our regular commentary here, I would also recommend the resources available on the new web page of the American Association for Public Opinion Research (AAPOR).

Danielle was right to recommend examining the verbatim text of survey questions and the way pollsters calculate their results. Where available, we include links to the most complete report of results in our "poll update" posts and in the tables that appear below the charts in our Polls section. The Polling Report also provides verbatim text for poll questions. Follow those links to see the full results and verbatim text when available.

However, we need to be careful to distinguish between tables or references in news stories that omits the results for the undecided category and those that reallocate undecided voters as Danielle describes.

In the US, virtually all polls are now reporting trial heat results based on calculations that include the undecided or "don't know" category. Some pollsters will put out "projections" based on their final poll before the election that use various methods to reallocate undecided voters (the Polling Report has a handy set of tables showing both "vote projections" and "final trial heats" from 2004 side by side). But I know of no pollsters putting out any such projections right now.

One way to check, if the undecided category is omitted, is to add up the results to each candidate. If they total 100%, then the results have almost certainly been recalculated.

More to the point, all of the results posted and plotted in the charts and tables in the Polls section of Pollster.com -- including the numbers discussed in yesterday's post -- are based on calculations that include the undecided category.

It is also important to distinguish between the undecided category on a trial heat question and separate questions that probe how certain or "decided" voters say they are. The undecided category tends to understate the real potential for change in a political contest. In recent weeks have written about questions asked on recent LA Times/Bloomberg and CNN/WMUR/UNH polls showing huge number of voters in Iowa and New Hampshire that are still less than certain about their choice.

Danielle may be thinking about computations that take those "uncertain" voters into account that Obama himself has been touting on the campaign trail. For example, NBC's First Read reported the following recent comments by Barack Obama at a rally in New Hampshire:

[Obama] also referred to a recent poll that had Sen. Clinton in the lead by twenty points, and said that it meant nothing besides what percentage of people were supporting which candidate. "Twenty percent of voters are with Sen. Clinton, 10 percent are with me, Edwards has about 5 percent and the rest of the candidates have less than that."

Citing that same poll, Obama said the most important fact overlooked by the pundits were the number of undecideds still up for grabs. "There are 55 percent of voters who are undecided. That's 55 to 60 percent of the people who are waiting to hear from you," he told the crowd of campaign volunteers made up of students from colleges in Massachusetts and across New Hampshire as well as local volunteers.

Obama was probably referring to the recent CNN/WMUR/UNH poll that gave Clinton a 22 point lead over Obama (with only 9% "undecided"), but also showed 55% of New Hampshire likely Democratic primary voters saying on a separate question that they are "still trying to decide" which candidate to support. If I set aside the uncertain 55% and recalculate vote preference numbers among those who say they are "definitely decided" or "leaning," I get 23% for Clinton, 10% for Obama and 6% for Edwards.

By Mark Blumenthal on October 5, 2007 7:06 PM | Permalink | Comments (2) | TrackBacks (0)

October 4, 2007

ABC/Post Poll: Surge or Outlier?

By Mark Blumenthal

About that new ABC News/Washington Post poll released yesterday, the one showing Hillary Clinton winning the support of 53% of "leaned Democrats" nationally to 20% for Barack Obama, 13% for John Edwards and single digits for the other candidates. Since this survey was the first to show a majority supporting Clinton, a lot of bloggers let loose with the adjectives, writing that she "pulls away" with a lead that is "crushing" or "explodes," telling us that we have reached "knockout punch time," that "she's killin' it," or declaring simply, "oh baby, when she moves, she moves." My friend Chris Cillizza parsed Clinton's "surge" in his Washingtonpost.com blog and concluded:

It doesn't take a polling expert to understand that Sen. Hillary Rodham Clinton's showing in the latest Washington Post/ABC News poll is a major moment in the campaign.

There is obviously no denying that the poll has created the perception of a major change in the race, but has a Clinton really experienced a "surge" of support over the last month, as indicated by the 12-point increase (from 41% to 53%) in her support on the Post/ABC poll? I'm not so sure.

I checked our national chart for the Democrats late yesterday and saw that this new ABC/Post poll appears to be very different from the others conducted over the last month. I quickly realized, however that this particular comparison highlights a frustrating limitation of our chart, which is that about half of the national polls plotted include Al Gore as an option and about half do not. Since Clinton typically picks up roughly five points on polls - including the new ABC/Post survey - that do not offer Gore as a choice, our standard chart may make the ABC/Post survey look like more of an outlier than it is.

So we decided to devote some time today to producing a chart that displays only the results for polls without Gore (using recalculated votes based on second choice when available). The result below plots 66 national polls released so far this year (about a third fewer than our standard version - as not all provide a "vote without Gore"):

The chart makes a few things clear. First, the 53% result for new Post/ABC poll is more of an "outlier" from the regression trend line than any poll conducted this year (it's the purple dot at the extreme top right of plot area). At 53%, the polls estimate of Clinton's support falls a full ten percentage points higher than our current estimate of the trend (42.5%) even without Gore in the race.

Second, the addition of the new poll has not budged our trend line. That is mostly by design. Professor Franklin set the sensitivity of the regression model to prevent the inevitable outlier from causing wild gyrations in the trend line. That is one of the reasons we prefer to use regression estimates over simple rolling averages.

One of the reasons we put so much effort into these plots is that they paint a picture of the random variation that is inherent in polling. You can actually see the random variation. While it typically appears as a random and predictable "cloud" of points, bigger outliers still occur from time to time. The more experience you have looking at poll numbers, the more you learn to understand that even with well designed polls, outliers happen. Our advice to poll consumers - as reflected in the design of the charts -- is to try to avoid over-reacting to any one poll.

Of course, we do not yet know whether this poll is really a statistical outlier. Other polls have obviously been showing a more gradual increase in her support recently, and it is still theoretically possible that Clinton's support suddenly lurched up ten points last week. Senator Clinton had appeared on all of the Sunday morning talk shows on September 23, just four days before this poll went into the field. Earlier that week sho also rolled out her plan to reform health care, an issue that ABC/Post and other polls confirm as one her great perceived strengths. But a sudden upward surge of this magnitude does not seem very likely, if only because no other news event so far this year has caused anywhere near as much change in the Democratic race.

We should know soon enough. Unfortunately, the AP-IPSOS survey released yesterday does not help resolve this issue, as Clinton's Sunday talk show tour de force occurred in the middle of their field period and (I'm told) after they had completed most of the interviews on the main sample.

So we will wait and see. But I'll wager that a month from now the real trend will not look nearly as dramatic as the one suggested by yesterday's news.

PS: It will probably take a few days, but we will have a full, regularly updating version of the national "vote without Gore" up soon.

UPDATE (10-16): More recent polls do move the Clinton trend line up and make the Post/ABC poll look much less like an outlier. More here.

By Mark Blumenthal on October 4, 2007 11:25 PM | Permalink | Comments (12) | TrackBacks (0)

POLL: InsiderAdvantage Republican Primaries in Iowa, New Hampshire, South Carolina, Florida, and Michigan

By Eric Dienstfrey

Five new InsiderAdvantage statewide surveys of likely Republican primary voters in Iowa, New Hampshire, South Carolina, Florida, and Michigan (conducted 10/2 through 10/3) finds:

• Among 1,339 likely caucus-goers in Iowa, former Gov. Mitt Romney leads former Mayor Rudy Giuliani (24% to 16%) in a statewide caucus; former Sen. Fred Thompson and former Gov. Mike Huckabee both trail at 13%, Sen. John McCain at 10%, and Sen. Sam Brownback at 5%.

• Among 1,822 likely primary voters in New Hampshire, Romney leads Giuliani (28% to 20%) while McCain trails at 17%, Thompson and Huckabee both at 8%, and Rep. Ron Paul at 6%.

• Among 1,281 in South Carolina, Thompson (at 21%) leads Giuliani, Romney, and McCain (all at 16%) while Huckabee trails at 11%.

• Among 725 in Florida, Giuliani leads Thompson (29% to 19%) while Romney trails at 16%, McCain at 10%, and Huckabee at 6%.

• Among 1,190 in Michigan, Giuliani edges out Romney (19% to 16%), while McCain trails at 15%, Thompson at 14%, Huckabee at 6%, and Paul at 5%.

• All other candidates receive less than five percent each.

We will add links once they are available.

By Eric Dienstfrey on October 4, 2007 10:29 PM | Permalink | Comments (7) | TrackBacks (0)

POLL: Gallup Dem Candidates

By Eric Dienstfrey

New analysis from the most recent Gallup national survey of 537 Democrats and those who lean Democratic (conducted 9/24 through 9/27) asks whether Sen. Hillary Clinton, Sen. Barack Obama, or former Sen. John Edwards would do the best job when faced with 17 different challenges as president.

The analysis finds:

• "While Clinton dominates on core policy issues, Sen. Barack Obama does relatively well on the handful of items included that tap into the candidates' ability to relate to people and heal divisions in the country. Democrats do not consider former Sen. John Edwards the best candidate on any issue."

• "The fact that Edwards receives his highest rating on the commander-in-chief dimension could say more about what Democrats think of Clinton and Obama on this issue, than what they think of Edwards."