Pollster.com: More on the ABC/Post Iowa Poll

November 20, 2007

More on yesterday's new Iowa poll from ABC News and the Washington Post. Stu Rothenberg has a scathing review of the coverage on Political Wire:

For years, Independent political analysts have been warning about reporters' tendencies to compare polls conducted by different polling firms, to over-interpret small changes in poll results and to treat the results of the most recent survey as if they are etched into stone. And yet that's what the two networks seemed to do.

[snip]

It's worth noting, though nobody did, that the July ABC News/Washington Post survey was dramatically different than other surveys taken at the time. This does not mean that the July ABC News survey was wrong or that the current one is incorrect. It is a reminder, however, that it's better to be cautious about reading too much into this, or any, poll -- even if you are paying for the survey.

The last Post/ABC poll did show Obama with a greater percentage of the vote than other surveys done at the same time. However, and I certainly agree with Rothenberg's general caution about comparing polls from different organizations too closely against each other. I'll have more on this when I finish (finally) the review of reports from the disclosure project (before Thanksgiving--I promise!). But the bottom line is that we have in Iowa almost as many different methodologies and conceptions of the potential electorate as we do polls. Those differences in method make for considerable variation in the results. In a sense, they're all outliers.

The best way to consider what "the polls" say about the Democratic contest is to look at our Iowa chart, though I would recommend focusing as much on the points (representing results from individual surveys) as the trend lines. Consider this screen grab from the 2007-only chart, which shows the Iowa results since late August for Clinton (purple), Obama (yellow) and Edwards (red). The trend lines draw on earlier data not seen in the snippet, but if you focus on the last month it is hard to see much of a trend from all the seemingly random noise.

11-20%20chart%20screen%20grab.png

By and large, the Clinton results have been slightly (but not consistently) better than the Obama results with Edwards generally trailing the two. Some of the differences stem from random noise, some from systematic differences in method. Which poll has been most "right" in recent weeks? We may never know. The best characterization, given the overlap in the ranges for each candidate, is the one that The Washington Post put on their own results: "The top three Democratic presidential contenders remain locked in a close battle in Iowa."

And speaking of methodology, ABC News polling director Gary Langer blogs again this week about the procedures they used to conduct this poll on his . His comments are worth reading along with Rothenberg's:

I blogged in August, at the time of our last Iowa poll, about our methodology there, and we followed the same random digit-dialing procedures this time. Again there's a lot of winnowing involved in getting down to likely voters: to get 500 likely Democratic caucus-goers we had to interview more than 4,800 adults in Iowa. That's a lot of calls.

Sampling methodology is a critical point of differentiation among surveys. Another difference is in the number of undecideds -- just 3 percent in our survey, vs. anywhere from 10 to 16 percent in other recently released Iowa polls.

-- Mark Blumenthal

November 20, 2007 in Divergent Polls, Iowa, The 2008 Race

Comments

Chuck Miller:

About the ABC/WP Poll...

It appeared the day after a report by Jeff Zelney in the NYT about the Clinton campaign in Iowa that highlighed many of the same points that were illustrated by the poll. If qualitative and quantitative information support one another, that leads me to believe both are accurate.

Posted on November 20, 2007 9:37 PM

____________________

Andrew:

David Gregory lied, according to the link to the Political Wire piece provided above, about the poll when he said "For the first time, Obama has a lead in that state (Iowa)".

As we all know, Obama has had the lead before in polls by Newsweek and a July ABC/WP poll.

Journalists are so excited at the prospect of Hillary losing the nomination that they exaggerate and lie in order to help bring her down.

Posted on November 21, 2007 4:13 PM

____________________

Bruce P:

Chris Cilizza's column today reported that this poll measured "electability" by the question "Which candidate has the best chance of getting elected president in November 2008?" and that the sampled voters rated Clinton higher than her rivals in this question. But this is not an electability question! It asks the sampled voters to handicap the entire 2008 election process, including the Democrtatic primaries. It's no surprise Clinton wins this question; the only thing being measured is the voters' awareness of the national polls. To measure electability they shoud be asking "Which Democratic candidate is most likely to defeat the Republican candidate in the general election?" Can you please let the people over at the Washington Post poll know that they screwed up?

Posted on November 21, 2007 10:53 PM

____________________

Nick Pangaakis:

Bruce-

I think what you are saying is that ABC's question "Which candidate has the best chance of getting elected president in November 2008?" could yield, for example, respondents who think Hillary's chances are 40%, Obama's 35%, etc. etc. all less than 50%; i.e., chances all less than even. Best chance doesn't mean enough to win and there is a difference.

First, I would tweak your question a little by adding "if any" to allow for a "none" response. Otherwise, the question could be considered leading.

Your question could be: "Which candidate, if any, is most likely to defeat the Republican candidate in the November general election?" [SHUFFLE NAMES"] But your question isn't much better. "Most likely" still doesn't mean "win" which I think is your issue.

How about "Which candidate, if any, is certain to defeat the Republican candidate in the November general election?" "Any others?" could be added because a win is a win. Or "Which candidate, if any, is certain to win..?"

If questionnaire length is not an issue, each candidate could be rated n a chances of winning scale "Certain, very good, not very good, or no chance at all" paying particular attention to the "certain" response in the analysis. Since questionnaire length is always an issue, this could be limited to three or four front-runners. However, Ron Paul response could be interesting. Winning isn't everything for some voters.

Questions and question order are two of the thorniest issues in polls and they should get more attention.

Nick Panagakis

Posted on November 25, 2007 3:19 PM

____________________

Bruce P:

Nick,

I guess my earlier comment wasn't clear. My issue isn't "most likely" vs. "win". It's "primaries + general election" vs. "general election" electability. What concerns the Democratic voters on the electability issue is which candidate can beat the Republicans (and vice-versa), not how subsequent primary voters will vote. The latter position wouldn't make any sense, given the strong influence the Iowa caucus exerts on those susbsequent primaries.

Posted on November 26, 2007 2:48 PM

____________________