Pollster.com

Articles and Analysis

 

Focus Grouping the YouTube Debate

Topics: Debates , Focus Groups

A few more thoughts about the various efforts to determine who "won" last night's CNN/YouTube debate. In addition to the SurveyUSA panel-back survey I blogged last night, there were (by my count) at least four different focus groups convened in various locations:

  • CNN reported last night on two "dial groups," one in Manchester, New Hampshire and another in Las Vegas, Nevada, in which selected voters constantly adjusted a dial that rates their reaction to whatever they are watching. (coverage here, here and here).
  • Dial group tester Rich Thau (Messagejury.com) did another dial group among 12 likely voters in Bedford New Hampshire. [Update: more on the Thau groups here, via First Read].
  • Fox News had Republican pollster Frank Luntz conduct what appeared to be a more traditional focus group (albeit with a larger than traditional group conducted on live television) in Charleston, South Carolina (transcript, video available from this page). [Update: A reader emails to say that the Luntz group also involved a "dial test"].

Because they are more qualitative, the "results" of these efforts can sometimes be in the eye of the beholder. Since they involve small, non-random samples of different kinds of voters, they frequently produce contradictory results. Some researchers like to use bigger groups (of 40-50 participants), but that approach increases the risk of creating a "group dynamic" that allows the most opinionated participants to sway others toward their point of view. We also know little from the various reports linked to above about the kinds of voters recruited for each group. How many were completely undecided before the groups? How many were political independents? The answers to those questions might tell us a lot about the reported outcome in each case.

Not surprisingly, last night's focus groups identified many different "winners." According to coverage from NBC's First Read, "Barack Obama got the most favorable [response] in terms of the best performance" in the CNN New Hampshire group, but their Nevada group showed "Bill Richardson and Hillary Clinton in a tie with six votes each." On the basis of favorable reaction from his focus group, Frank Luntz concluded that "Obama will be shown as the winner of tonight's debate."

Meanwhile, depending on which source we turn to, the Rich Thau group of 12 New Hampshire voters produced good news for John Edwards, Barack Obama and Joe Biden. Under the headline, "Very Much an Edwards Night," Thau reports that "a plurality of our group (four of 12) thought Sen. Edwards won the debate." However, NBC's First Read tells us, "his survey had Obama doing the best (in terms of improvement from pre-debate to post-debate; Obama and Edwards tied with the highest post-debate score)." And Thau's before-and-after "comfort scores" shows the biggest gains by Joe Biden.

If you're having trouble making sense of all this, you're not alone.

Also, I am guessing that the audience for the YouTube debate, while probably larger than usual for CNN, was still a tiny sliver of those who will ultimately vote in Democratic primaries or caucuses. Assuming the ratings followed typical patterns, we should assume that far more voters saw or read news accounts about the debate this morning than experienced it in full last night.

In that regard, consider this very sensible observation made last night by Chris Bowers, on the liberal site Open Left:

I noticed that both on CNN, and in the progressive blogosphere, several people started trying to determine who won, and even making declarations that Candidate X had, in fact, clearly won the debate. I was taken aback. There was a winner? There was even a competition? What did the winner, well, "win?" New hard-core voters in the Iowa and New Hampshire primaries? New donors? The deliverance of a biting comment that will psychological scar one of his or her opponents so badly that s/he cannot continue on the campaign trail with naught but a thoroughly shredded sense of self-esteem?

The truth is, there is only one objective way for any candidate to "win" the debate: if the debate, or the post-debate spin, cumulatively result in your campaign moving closer to taking the nomination than you were before the debate. That's it. From this "God's eye" perspective, it is doubtful that anyone won the debate. The factors that make the most difference in pushing someone closer to the nomination are, first, new Iowa supporters and, second, new supporters in New Hampshire. Everything else is pretty distant, and I seriously, seriously doubt anyone picked up much, if anything, in those two categories

That sounds about right.

 

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR