Pollster.com

Articles and Analysis

 

Debate Reaction: What's a Win?

Topics: Barack Obama , Debates , Internet Polls , John McCain , Knowledge Networks

So who won the debate? We now have results from four five** surveys, plus one "dial" focus group, and they all show essentially the same thing. On the question of who "did a better job," respondents rated Barack Obama better than John McCain. However, the surveys went about their measurement tasks in different ways, and we will not know for a few days whether the stronger impression that Obama made will into changes in vote preference.

Before turning to the data, it is worth thinking about what we are trying to measure. What does "winning" a debate mean? Is it about which candidate "did a better job" as perceived by the voters? Or is it about which candidate made the most progress in growing or solidifying their support? While most of the pollsters have emphasized their results on the "who did a better job" question, what most of us want to know is whether the debate made a difference in overall vote preference. That latter issue is much harder to gauge from these first "snap" surveys.

So on to the data:

  • USA Today and Gallup called 1,005 randomly sampled adults on Saturday and found 63% who said they had watched the debate. Of these, 46% thought Obama "did a better job" in the debate, 34% preferred McCain.
  • CNN/Opinion Research Corporation re-contacted 524 adults Friday night who watched the debate. Their respondents had indicated on a previous random sample telephone survey that they planned to watch the debate. Of these, 51% thought Obama "did the best job in the debate," and 38% thought McCain did best.
  • CBS News and Knowledge Networks re-contacted 483 previously interviewed, uncommitted voters (those "who don't yet know who they will vote for, or who have chosen a candidate but may still change their minds") on Friday night that had agreed tow watch the debate. Their respondents were members of the Knowledge Networks nationally representative internet panel (more on that below). Thirty-nine percent (39%) thought Obama "did the best job -- or won," while 24% preferred McCain.
  • Zogby International surveyed 2,102 respondents from their non-random online panel. They "gave Obama the win by the slightest of margins" (47% to 46%).
  • Democratic affiliated Democracy Corps conducted a "dial" focus group involving 45 undecided voters in St. Louis, Missouri. Thirty-eight percent (38%) of this non-random sample thought Obama won, 27% gave the win to McCain.

The critical limitation of the "who won" question is that debates almost always reinforce voters' pre-existing preferences. Those who tune into the debate with their minds made up (or mostly made up) tend to rate their preferred candidate as the "winner." If the debate audience prefers one candidate by a lopsided margin, that candidate will usually come out as the perceived winner, so it is important to try to look at these results either by party or by the candidate previously supported.

Both Gallup and CNN/ORC found more self-identified Democrats than Republicans among the debate viewers, mostly reflecting the party identification advantage that Democrats hold nationwide. The CNN story, for example, tells us:

The results may be favoring Obama simply because more Democrats than Republicans tuned in to the debate. Of the debate-watchers questioned in this poll, 41 percent of the respondents identified themselves as Democrats, 27 percent as Republicans and 30 percent as independents.

The story also quotes CNN Senior Political Researcher Alan Silverleib saying that the debate amounted to a tie "especially after accounting for the slight Democratic bias in the survey." "Given the direction of the campaign over the last couple of weeks," he adds, "a tie translates to a win for Obama." Unfortunately, the CNN story did not include a cross-tabulation by party, and made no reference to the reaction of independents.

The Gallup analysis, however, tells us (with emphasis added):

The data show a predictable pattern of response from Republicans and Democrats. Seventy-two percent of the former and 74% of the latter said that their party's candidate did the better job in the debate. This reinforces the conventional wisdom that many viewers watch a debate through their preexisting perceptual framework and end up with nothing more than reinforcement for what they believed before the debate began. But among the crucial group of independents who watched the debate -- those most likely to actually be swayed by what transpired, Obama won by 10 points, 43% to 33%.

For my money, the most useful analysis so far is the one from CBS and Knowledge Networks, because they focused on previously uncommitted voters and were able to report on how the debate changed specific impressions among these critically important voters. They found, for example, that Obama gained 16 points on being "prepared for the job of President" (from 44% who answered "yes" to 60%), while John McCain gained only five points (from 36% to 41%) on "understands the needs and problems of people like you."

The CBS/KN survey also found a modest shift in Obama's direction on vote preference. Before the debate Obama had a two-point lead (36% to 34%) among the uncertain voters. After the debate, Obama led by 12 points (41% to 29%).

Of course, how much credence you give these results may depend on what you think of the Knowledge Networks panel. CBS News has been using it for eighty years to measure the reactions to debates and speeches, because of its unique design. Knowledge Networks recruits members to its panel using the same "random digit dial" telephone sampling procedures that most pollsters use to conduct national surveys. Those without internet access are provided with Web TV service so that they can participate in the surveys.

While Knowledge Networks has good reason to say they conduct "scientifically valid" online research, we should keep in mind that like any survey panel, their approach has limitations. Keep in mind that the 483 respondents selected randomly from the KN panel had to agree to participate four times: First, when they were first contacted by telephone; second, at the end of that interview when they agreed to join the panel; third, when asked to participate in the pre-debate survey for CBS and fourth on the post-debate survey.

So the true response rate would be much lower than the cold-call national surveys we typically plot on our national trend chart. A lower response rate does not, but itself, introduce error. In this case, it simply argues for a bit more caution when interpreting the results.

Back to the Gallup report. They also include a paragraph that makes reference to their historical archive of comparable "quick reaction polls" conducted over the years:

History shows that "winning" the first presidential debate does not necessarily translate into winning the election. Ross Perot, Al Gore, and John Kerry are among those who were seen by debate watchers in quick reaction polls as having done the better job in the first debate of their campaign year, and all eventually lost their elections. There are two presidential (and one vice presidential) debates yet to come, and much can change. The most important indicator of the impact of the debate may be trends in overall candidate support, where, at the moment, Obama leads McCain.

I made a similar argument in my column this week, and I think the bottom line still holds. If "winning" the debate means gaining electoral support (or perhaps, in Obama's case, solidifying his national lead), the true test will come from the nationally representative surveys of all voters conducted over the next few days. So if we're willing to wait a few days, we will know for sure.

**Update:  I missed the LA Times/Bloomberg survey that interviewed 448 registered voters Friday evening through Sunday who said they watched the debate.  From the LA Times article:

Overall, more voters in the Times/Bloomberg poll -- 34% -- thought the debate was a draw than believed either candidate had prevailed. Thirty-three percent of debate watchers said Obama did the best job, and 29% gave the nod to McCain.

These results, noted by Marc Ambinder, are intriguing, although it would be helpful to know the partisanship of the debate watchers surveyed:


The poll also indicated that the younger, less-experienced Obama has made strides since last week in convincing Americans that he can handle the toughest challenges facing the country, including the economy and international affairs.

Obama was seen as more "presidential" by 46% of the debate watchers, compared with 33% for McCain.

The difference is even more pronounced among debate watchers who were not firmly committed to a candidate: 44% said they believed Obama looked more presidential, whereas 16% gave McCain the advantage.

 

Comments
Morssa:

Hey, can you please post the new polls? It seems you stoped doing that 2 days ago. Thank you.

____________________

DemocratAbroad:

Off topic, but as one of the 7-10% of the color blind males on the planet, I want to thank you for using colors on the electoral map that everyone can read.

____________________

"CBS News has been using it for eighty years"

Eighty years, eh? I think we might have an order of magnitude issue here...

Seriously though, I'd be interested in hearing your thoughts Mark on the validity of the Knowledge Networks panel. Josh Clinton over at Princeton has a working paper in which he finds no evidence of systematic bias in their samples, but I'm not so sure.

____________________

Loyal:

Forgive my ignorance here. Why are folks suggesting within +/- 0.5 points as the range of no interest? To me, the definition of menaingful change has to be tied to the distribution of the poll. Is this a polling thing, since I come from a different tradition of numeracy.

Loyal

____________________

Loyal:

Oops. I posted the above on the wrong thread. Never mind! Sorry.

Loyal

____________________

Gary Kilbride:

The so-called "quick reaction" polls may have favored Gore in 2000, but that's misleading beyond description. I had about 20 debate watchers in my living room and I knew otherwise long before the debate was over. The apolitical women, in particular, were aghast at Gore's sighs. One of them called her mother in hysterics; "Mom, I can't believe I was going to vote for him!"

The immediate network analysis of that debate was the most bizarre thing I've ever seen. It's like they had read the debate and not watched it. I held my breath and hoped somehow my group was not representative, but deep down I knew better. A day later the CNN daily tracking poll indicated a major shift toward Bush, and I didn't need to ask why.

Perot and Kerry aren't ideal reference points either, Perot with no chance to win and Kerry far behind an incumbent at that point, his largest deficit of the campaign. The first and only debate was certainly vital to Reagan, and also to JFK among four, although that was before my time.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR