Pollster.com

Articles and Analysis

 

Riehle: Just Don't Do It

Topics: CNN , Instant Reaction Polls , Speech Reaction

Today's Guest Pollster article comes from Thomas Riehle, a Partner of RT Strategies.

Technological capabilities can become temptations to conduct research studies that add nothing to our knowledge of public opinion, just because we can. Get thee behind me, Satan!

For example, it would be no problem, technologically, to display squiggly lines with the moment-by-moment reactions of a panel of viewers to the blathering of the talking heads on news show panels. The Onion demonstrates what a mess that would be, in a parody entitled "New Live Poll Allows Pundits to Pander to Viewers in Real Time."

What would happen if we let the talking heads see whether viewers at home agreed or disagreed with what they were saying, "using the Insta-Poll Tracker on our web site"? The talking heads would become self-conscious about the direction of their own squiggly line and start tailoring their statements...word by word...to make the squiggly line go up.

Insta-polls like September 9th's CNN/Opinion Research Corporation poll of adults who watched President Barack Obama's address to Congress may have a similar effect on poll respondents. Mark Blumenthal correctly points out the age-old problem of such polls--the partisan make-up. Last night, the audience for this address was heavily weighted with Obama supporters rallying to watch their leader, supplemented with a few civic-minded Americans who would watch any Presidential address, regardless of their own partisanship. Of the 427 adults in this study, all of them interviewed September 5-8 in advance of the speech, and all of whom indicated both an intention to watch the speech and a willingness to be re-interviewed after the speech, 18% were Republicans, 45% Democrats. These kinds of post-speech poll samples always skew heavily in favor of the speaker. Pollster.com's report on this poll last night squeezes out what knowledge can be gleaned by comparing the "bump" among this group of speech watchers to the bump registered among similarly situated groups of speech watchers in the past.

The problem with this kind of insta-poll may be exacerbated when the study is designed, as this one was, to compare the pre-speech responses of speech watchers to opinions after the speech. In the pre-speech survey, I would guess that respondents would strive to express their opinions as forthrightly as possible, as most survey respondents do. In the follow-up poll after the speech, however, I am afraid respondents would be like the Onion's self-conscious pundits. They'd be aware that they are about to become as much a part of the story as South Carolina Republican Rep. Joe Wilson who heckled the President. They'd tailor their answers to make their leader look good. Drawing much of a conclusion from their answers would not be any fairer than judging the entire Republican caucus by the boorishness of a few Members.

 

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR