Pollster.com: Open-Ends on the Dems

December 07, 2006

I wrote earlier this week about an open-ended presidential preference question from Gallup. In another free-for-today-only analysis, Gallup's Lydia Saad provides results from a different set of open-ends -- questions that provide no answer categories and allow respondents to answer in their own words -- on the four best known Democrats mentioned as potential candidates. The results show that a majority of Democrats have good things to say about Hilary Clinton and Barack Obama, fewer are positive about Al Gore and nearly half of Democrats react negatively to John Kerry.

Gallup used their national panel survey to ask a simple question about each of the four: "What comes to your mind when you think about [name]?" The Gallup interviewers wrote down the verbatim responses, and then the Gallup analysts "coded" those responses, assigning each to a category (such as "experienced," "strong," "like him/her," etc.). They then classified each of the categories as positive, negative or other. The report provides results to those classifications, both among the 1,003 adults and among an unspecified subgroup of those who think of themselves as Democrats (presumably about 350 interviews). The results among Democrats are as follows:

12-07%20gallup%20open%20end.png

This quick post does no justice to the level of detail available in the full analysis, but some of the most obvious findings:

Comments about Hilary Clinton are net negative among all adults (37% to 50%) but nearly two-to-one positive among Democrats (56% to 31%).

The positive comments about Clinton among Democrats are mostly about her leadership credentials (qualified, experienced), strength and intelligence.

The independents and Republicans that dislike Clinton are apt to say they dislike her personally, Democrats are more likely to worry that she is not electable (5%) or that she is "riding Bill's coattails" (10%).

When asked about Barack Obama, more than a third of Americans (38%) and more than a quarter of Democrats (28%) are unable to offer anything specific that comes to mind.

Comments about John Kerry are overwhelmingly negative. Even among Democrats, 49% offer negative comments compared to only 28% that have something positive to say.

Of course, the sample of Democrats is relatively small and represents, at best, all Democratic identifiers nationally -- not likely Democratic primary voters and certainly not likely voters or caucus participants in early states like Iowa, New Hampshire, Nevada or South Carolina.

It is also worth noting that these results are from a Gallup "panel" survey. The methodology blurb at the end of the analysis tells us that "respondents were randomly drawn from Gallup's nationally representative household panel, which was originally recruited through random selection methods." Generally speaking, a panel is a pool of respondents who agree to regularly participate in opinion surveys. When Gallup says their panel is "nationally representative," it presumably means that panel members were recruited using a conventional telephone survey and a random-digit-dial (RDD) sample.

What we do not know (unless Gallup has posted a more detailed description of their methodology elsewhere that I have overlooked) is the size of the panel, the typical response rate, how Gallup weights or statistically adjusts the results and how often panel members agree to be interviewed. The two key issues with any panel survey -- even in we assume that the pool of potential respondents is a representative random sample -- are (1) whether a willingness to be interviewed more than once allows an additional bias to creep into the sample and (2) whether the experience of being interviewed changes the respondent. On the last point, assume that respondents complete a Gallup survey on issues in the news. Do those individuals subsequently pay more attention to issues in the news and become better informed?

I raise all of this, in part, because National Journal's Hotline yesterday reported that party identification for this Gallup panel survey is 35% Democratic, 36% Republican and 29% independent or other. The Republican percentage (36%) is considerably higher than on recent national Gallup surveys. Party identification among all adults has averaged 35% Democrat, 28% Republican on Gallup's last five national surveys conducted in October and November, and 35% Democrat, 30% Republican on all surveys conducted since July.

Of course, as discussed often both here and on Mystery Pollster, Gallup does not weight by party, so random variation in the results for party ID is to be expected. Still with a panel based sample, it is a good idea to keep an eye on this sort of variation.

-- Mark Blumenthal

December 07, 2006 in Measurement Issues, The 2008 Race