Mark Blumenthal | September 10, 2007
Topics: 2008 , Disclosure , The 2008 Race
Before my vacation in August, I used two posts to review the sorry state of disclosure with respect to how tightly (or not-so-tightly) the pollsters screen for likely primary voters, especially those in early primary or caucus states. Today I want to take a look at another question we might want to ask about these polls to help sort out the differences in results among them: What are the demographics?
The issue that becomes much more acute in pre-primary surveys is that the pollster is trying to determine two things at once: (1) identify the voters that will participate in the primary and (2) measure the attitudes and vote preference of those voters. When different primary polls produce seemingly contradictory results, the culprit is usually the people selected. So if we want to try to tease out the reasons why polls show different results, we want to know as much as we can about the "likely voters" they sample.
Just last week, for example, I looked at the differing results in some polls of Iowa Democratic "likely caucus goers" and found wide variation in the number reporting past caucus participation. As explained in the post, that variation appears related to support for various candidates (previous caucus goers are more likely to support John Edwards, newcomers more apt to support Hillary Clinton and, to a lesser extent, Barack Obama).
For the Democratic candidates, it is not hard to imagine that variation in demographic variables like gender, age and race might have similar effects. For example, various public polls have shown that Clinton does better among women and Obama better among younger voters. Clinton and Obama also dominate among African Americans to the relative detriment of John Edwards and other candidates. Demographic patterns in polls of Republican primary voters have been relatively inconsistent, although Giuliani and McCain tend to do better among moderates and Republican leaning independents.
The demographic composition can vary widely. Consider the African American percentage of the likely Democratic primary electorate in South Carolina in polls released over the last several months.
- 56% - Hamilton-Beattie (D) and Ayres-McHenry (R), 4/14-19/2007
- 53% - Garin-Hart-Yang (D), 4/9-12/2007
- 50% - Clemson University Palmetto Poll, 8/20-24/2007
- 43% - Public Policy Polling (D), 5/31/2007
- 40% - Public Policy Polling (D), 8/13/2007
- 40%* - CNN/ORC, 7/16-18/2007 (percentage based on black sample size, may have been unweighted)
- 30% - Chernoff Newman/MarketSearch, 4/9-16/2007
Needless to say, the variation above is huge (although the relationship between racial composition and the Clinton-Obama result has been weak so far). While these results certainly cannot all be right, the right answer is not obvious. African-Americans comprised 47% of Democratic primary voters in the 2004 South Carolina primary, according to the network exit poll, but of course an exit poll is also a survey with potential problems of its own. And whatever the past result, the composition in 2008 may be different.
The important point is that educated poll consumers will want to know all they can about the demographics of pre-primary polling, and unfortunately, such information is very hard to find. I went back to the public releases from 23 different organizations that have released public polls in the last six months or so in Iowa, New Hampshire and South Carolina.
- Hamilton Beattie/Ayres McHenry (SC) - gender age, race, income, party
- PPP (IA, SC) - gender, age, race (SC only), party
- SUSA (NH) - gender, age, ideology
- Time (Iowa) - gender, age, education, race, party affiliation, percentage of past caucus goers
As I recall, the Garin-Hart survey of South Carolina also included some demographics (as I included their result for race in my post on 5/3), but their PDF release is no longer available online. A few organizations provide more limited information: The American Research group routinely provides the percentage of independents included in their samples (but not standard demographics). The ABC/Washington Post survey of Iowa caucus goers provided results to a question on past caucus attendance (but nothing more). And two surveys of South Carolina - from Clemson University and CNN - both provided the African-American composition only.
The national surveys are not much better. Only two organizations routinely provide full data on the demographic composition of the subgroups that hear primary trial-heat questions: Cook Political Report/RT Strategies and Diageo/Hotline.
For more than a month, I have been promising some ideas about what we might do about this paucity of information. I'll have more on that in the next post.