Eric Dienstfrey | March 1, 2008
Clinton 47, Obama 43
Clinton 47, Obama 43
American Research Group
Obama 45, Clinton 43... McCain 54, Huckabee 31, Paul 7
Clinton 45, Obama 45... McCain 58, Huckabee 23, Paul 8
Fleming & Associates/Eyewitness News
Clinton 49, Obama 40
Last fall, I asked all of the organizations that conducted Iowa Caucus surveys to disclose the demographic composition of their samples and several other aspects of their methodology. Although cooperation was mixed, our "disclosure project" demonstrated how polls that are theoretically reporting on the same population of "likely voters" can sample very different kinds of people.
With so much attention focused on Ohio and Texas this week, I thought it would be worthwhile to attempt a less ambitious version. Earlier this week, I sent all the pollsters that had released surveys in either state in recent weeks to disclose some of their demographic composition and to estimate the percentage of Texas adults that their surveys represent.
While new polls have been appearing every day, I want to report the responses so far, starting with Texas.
The demographic mix is especially important in Texas given the large percentages of both African American and Latino voters there. Fortunately, in this case at least, we now have a fairly complete look at how these polls of "likely Democratic primary voters" differ demographically. When the data was not already in the public domain, I received quick cooperation (in Texas) from the pollsters at Washington Post/ABC News, Constituent Dynamics, Hamilton Campaigns and Public Policy Polling (PPP). Also, an encouraging number of pollsters have included demographic profile data their Texas releases, including several that are typically more reticent, including ARG, Rasmussen Reports. And thanks to the Houston Chronicle, even the Zogby/Reuters/C-SPAN poll helped make the world a "better place" by making cross-tabs featuring demographic composition available on Chron.com.
As in Iowa, the results show considerable variation, particularly on the Latino or Hispanic percentage of the samples, which vary from a low of 24-26% (ARG) to a high of 39% (Post/ABC). Other categories also show wide variation including the percentages of African Americans (from 14% to 23%), women (from 51% to 58%) and voters over 65 years of age (from 15% to 26%
30%; comparisons by age categories are especially difficult, since no two pollsters report exactly the same age breaks [Update: At "Joe's" suggestion, I've added some additional age breaks where available]).
I have included comparable numbers from the 2004 exit poll,* although we will not know what the "right" answer is until the votes are cast and results from this year's exit poll are available.
If they had not yet done so in their public release, I also asked pollsters to estimate the percentage of Texas adults represented by their samples, which is a decent measure of how tightly they screened for likely voters. The percentage of eligible adults that participated in the Texas Democratic presidential primary was just 6% of eligible adults in 2004 and 9% of eligible adults in 2000 (if calculated as a percentage of all adults, including non-citizens, the percentages would be 5% and 8% respectively).
Even though the following table includes results for just four pollsters, the range of adults represented** is huge, from a low of 7-8% for the Texas Credit Union League(TCUL)/Hamilton Campaigns/Public Opinion Strategies poll to a high of 40% on the first poll from SurveyUSA. The TCUL poll obviously comes closest to past turnout, although turnouts have been much higher in other states so far this year than in 2004. The ABC News release concedes that an actual turnout of 24% of adults is "unlikely" but reports that "vote preference results are similar in likely voter models positing much lower turnout."
Next, Ohio... and then after posting all the statistics, I'll come back and speculate about what they might mean for what everyone cares about: where the race stands heading into the final weekend. Given the time crunch, I put these tables together quickly. So if you spot a typo or can help fill in a blank that I've missed please send an email (to questions at pollster dot com).
*UPDATE: All of the 2004 exit poll results in the table above are from the final weighted data available from the Roper Center archives. Some of the percentages differ slightly from those posted on election night 2004 by CNN and still available online. The difference is likely due to final weighting done after 10:43 p.m. on March 9, 2004, the time the CNN tables were last updated. An earlier version of the table posted above was based, in part, on the CNN results.
**For the ABC/Post, CNN and SurveyUSA polls, we estimate the percentage of adults represented by dividing the number of interviews conducted among likely primary voters by the number of adults interviewed. Since those samples of adults will include non-citizens, and since non-citizens are 12% of the Texas adult population, I calculated a range for the two polls -- PPP and TCUL -- that sample from lists of registered voters (see "Update II" of this post for more explanation).
[Table updated on 3/1 to include Below/WFAA/Public Strategies surveys].
Clinton 47, Obama 45
Obama 48, Clinton 45
Clinton 46, Obama 38
I am playing catch up after a day spent mostly in airports and offline. So I pass along this item after only a brief review: SurveyUSA's Jay Leve has posted yet more on their new and evolving pollster report cards. It follows up on our previous exchange and my column posted earlier today.
The gist of this post is that, in at least one example -- polls conducted prior to this year's Iowa caucuses -- the "most accurate" poll was not necessarily the last one. Anyway, our always attentive readers will be able to give Leve's post more attention than me today, so please have at it.
Meanwhile, you may want to click through to Leve's post if for no other reason than he kindly posted a photo that, in all the craziness of the last few months, I never managed to post here: It shows Charles Franklin and me receiving the AAPOR Warren Mitofsky Innovators Award last May (that's Franklin on the left, me on the right and Mia Mather, Warren Mitofsky's widow, in the center).
American Research Group
Pew Research Center
We encourage our readers to click through to see field dates, sample sizes, margins of sampling error, target populations and addition results.
Clinton 47, Obama 43
My latest column for NationalJournal.com, which discusses measuring a survey's merit, is now online.
Obama 48, Clinton 43... McCain 61, Huckabee 24, Paul 3
Middle Tennessee State University
Public Policy Polling (D)
Texas's 14th Congressional District
TX-14 Republicn Primary
Paul 63, Peden 30
US President Rep Primary
McCain 49, Huckabee 27, Paul 18
Obama 48, Clinton 44
Clinton 46, Obama 42
McCain 68, Huckabee 17, Paul 5
Civitas Institute DecisionMaker Poll
I am crashing on a deadline for my National Journal column, but want to pass along the analysis just posted by Gallup about the differences in their recent polls on the Democratic nomination. Here is the gist:
Gallup probably does more national polling than any other firm does, so this is certainly not the first time and won't be the last time that one of our polls produces results inconsistent with other polls. When this does happen, though, we do our best to analyze why those differences might have occurred. Unfortunately, these investigations rarely yield a proverbial smoking gun, and, beyond normal sampling error, it is not clear why one poll might differ from other polls -- even in cases when the differences between polls appear to be beyond the margin of error. This appears to be the case with Gallup Poll Daily tracking and USA Today/Gallup poll as well.
Having read my two posts on this subject earlier this week, a pollster friend of mine emailed to ask, "in spirit of Occam’s Razor, am I correct in simply saying 'they don’t have a clue why there is a difference?'"
That's about right.
To their credit, Gallup's analysis thoroughly reviews the subtle methodological differences between the two surveys, and includes a spreadsheet comparing the demographic composition of the two samples. That is more than we can say for some other pollsters. Still, their bottom line is uncertainty:
To reiterate, none of the known differences between Gallup Poll Daily tracking and the USA Today/Gallup poll, based on our analysis, are obvious causes for the disparity in the Democratic ballot estimate between the two samples. In fact, the estimates of the Republican horse race between the two samples were almost identical.
I will let our always throughout readers chew this new analysis over while I go back to drafting my column.
Obama 48, Clinton 42
McCain 46, Clinton 40...McCain 44, Obama 42
Public Policy Polling (D)
Ohio's 10th Congressional District
Kucinich 55, Cimperman 29, Ferris 5...
First, reader s.b. notes:
[W]ith an automated survey, if its in English, they aren't sampling spanish only or mostly spanish speakers. I think it skews these results.
Some pollsters (such as Gallup) offer voters the opportunity to complete the survey in Spanish when they encounter Spanish speaking respondents. Most pollsters, however, will simply end the interview in these instances. I asked SurveyUSA's Jay Leve about their procedure in Texas and he notes that while they do have the facility to offer respondents the option to complete a survey in either English or Spanish (and have done so in mayoral elections in New York and Los Angeles and some congressional districts), they did not offer a Spanish interview for their Texas poll.
However, before leaping to conclusions about the SurveyUSA results, keep in mind that
none only one of the other Texas pollsters report using bilingual interviewing for any of their surveys [Correction: interviews for the Washington Post/ABC News poll "were conducted in English and Spanish"]. Three of the other pollsters -- Rasmussen Reports, PPP and IVR polls -- also interview with an automated methodology rather than live interviewers.
And before leaping to conclusions about all the Texas polls, we might want to know just how many Latino voters in Texas speak only Spanish. I have not done survey work in Texas, but my memory from conversations with pollsters that do is that the percentage that will actually complete an interview in Spanish when offered is typically in the low single digits.
Second, several commenters have speculated about the small changes in the demographic composition of the last two SurveyUSA Texas polls. For example, "Mike in CA" points out:
Hispanic turnout at 28% sounds just about right. The last SUSA survey had it at 32% which was way too high. It seems SUSA has scaled back their Hispanic estimates, so they must have a reason. Additionally, the boosted AA to 23%, from 18%. Seems reasonable considering the extraordinary increases in early voting turnout from Houston and Dallas [emphasis added].
That's not quite right. Keep in mind that SurveyUSA's approach to likely voter modeling is comparable to that used by Iowa's Ann Selzer, in that they do not make arbitrary assumptions about the demographic composition of the likely electorate. As SurveyUSA's Jay Leve explains, they "weight the overall universe of Texas adults to U.S. census" demographic estimates, then they select "likely voters" based on screen questions and allow their demographics to "fall where they may." So some of the demographic variation from survey to survey is random, but large and statistically statistically significant variation should reflect real changes in the relative enthusiasm of voters. Leve goes into more detail in the email that I have reproduced after the jump, which also includes the full text of the questions they use to select likely voters.
A quick follow-up on my post last night about the conflicting results from Gallup on the Democratic presidential race as measured by two different national surveys: The latest USA Today/Gallup survey and the entirely separate Gallup Daily tracking. I exchanged email with Gallup's Frank Newport and can clarify two issues.
First, Newport promises further analysis on the difference between the two surveys, perhaps later this afternoon. We will link when available.
Second, the two Gallup surveys differed slightly in the the populations that were asked the primary vote preference question (which means that I assumed wrong in a comment I posted in the previous entry). The USA Today/Gallup poll reports Democratic presidential preference of all adult Democratic identifiers and "leaners" (those that initially identify as Democrats but say on a follow-up question that they "lean" Democratic). It does not screen for registration or vote intent.
As explained by Gallup's Jeff Jones in a post here three weeks ago, the Gallup Daily tracking screens further for adult partisans who also say they have voted or intend to participate in the Democratic primary or caucus in their state. They include:
Republicans or Republican-leaning independents who say they are extremely, very or somewhat likely to vote in their state's primary or caucus when it is held.
Democrats or Democratic-leaning independents who say they are extremely, very or somewhat likely to vote in their state's primary or caucus when it is held.
We [also] make provisions for those residing in states that have already held their primary caucus - those who indicate they have already voted are considered extremely likely to vote, and those who did not vote in their state's primary or caucus would be excluded from the base.
However, Newport adds that the different screen "would not by any means in and of itself account for the differences in the results between the two polls." Presumably, their upcoming analysis will elaborate.
Third, with the help of Gallup releases sent to my National Journal colleagues, I can report on the questions that preceded vote preference on the USA Today/Gallup survey:
First, we have some questions about the election for president, which will be held in November; that is, in November 2008.
1A. How much thought have you given to the upcoming election for president -- quite a lot, or only a little?
2. Do you approve or disapprove of the way George W. Bush is handling his job as president?
3. Next, we'd like to get your overall opinion of some people in the news. As I read each name, please say if you have a favorable or unfavorable opinion of these people -- or if you have never heard of them. How about --
[ITEMS A-C RANDOM ORDER, THEN ITEM D ASKED, THEN ITEMS E-G RANDOM ORDER]?
A. Hillary Clinton
B. John McCain
C. Barack Obama
[Note: E-G were omitted from the release]
4. (Asked of Republicans and independents who lean to the Republican Party) Next, I'm going to read a list of people who are running in the Republican primary for president. After I read all the names, please tell me which of those candidates you would be most likely to support for the Republican nomination for president in 2008, or if you would support someone else. [ROTATED: Former Arkansas Governor, Mike Huckabee; Arizona Senator, John McCain; Texas Congressman, Ron Paul; former ambassador, Alan Keyes]
5. (Asked of Republicans and independents who lean to the Republican Party) Which comes closer to your view about Mike Huckabee’s campaign for the Republican nomination for president-- [ROTATED: He should drop out of the race, (or) he should continue his campaign]?
6. (Asked of Democrats and independents who lean to the Democratic Party) Next, I'm going to read a list of people are running in the Democratic primary for president. After I read all the names, please tell me which of those candidates you would be most likely to support for the Democratic nomination for president in 2008, or if you would support someone else. [ROTATED: New York Senator, Hillary Clinton; Former Alaska Senator, Mike Gravel; Illinois Senator, Barack Obama].
ABC News polling director Gary Langer blogged on the conflicting national results, and he points out that "question order always is a possible culprit, but it doesn't usually make for differences like these." True, though it would be useful to know what questions precede the vote preference on the Gallup Daily tracker.
Langer also concludes with this bit of advice worth repeating about the four recent national polls (including one from CBS/New York Times showing Obama ahead by 16 points and one by AP/IPSOS showing Obama with a nominal 3 point lead):
For the moment, with the cause of these differing estimates up in the air, when considering the two national polls that show Obama ahead it would be prudent also to keep in mind the two that show the race essentially tied.
Note: The Gallup Daily Tracking survey is a different sample than the USA Today/Gallup survey, which shows Obama leading Clinton.
Public Policy Polling (D)
Obama 54, Clinton 38
Obama 50, McCain 38... Clinton 46, McCain 46
In the midst of a very busy day, the polling mystery de jour is certainly the dueling national Democratic primary results from Gallup. The new USA Today/Gallup poll out this afternoon (and conducted Thursday to Sunday, 2/21-24) shows Barack Obama leading Hillary Clinton nationally by 12 points (51% to 39%), while the latest Gallup "Daily" tracking release (conducted Friday to Sunday, 2/22-24) shows Obama edging Clinton by just two points (47% to 45%).
Our comments section and my email are overflowing with questions about why that may be. The answer is not obvious. Gallup editor-in-chief Frank Newport devotes an "editor's blog" item to this question (hat tip to Brian Schaffner who posted a link in the comments). Newport seems to be stumped:
There are some differences in the methodology between these two Gallup polls, including different days of interviewing and some differences in question order. But generally speaking, when Gallup conducts separate polls measuring the same variables at roughly the same times, the estimates are usually and predictably quite similar.
Sampling differences and the impact of random factors inherent in the survey process can sometimes explain why two polls are different. Given the similarities in the two polls' Republican estimates, however, I think the Democratic differences may well be another indicator of the conflicted nature of the Democratic race this year. It's not an easy choice, and as pre-election polls have shown in reference to the actual vote in some of the primary states, there is a lot of volatility out there among Democrats.
I have written quite a bit about the "hard choice" that Democrats face, and you can see continuing evidence of it in today's new Texas poll from CNN/ORC. Among likely Democratic primary voters in Texas, both Barack Obama and Hillary Clinton enjoy 80% favorable ratings. However, the mere potential for "volatility" does not explain as big a difference as is evident on the two Gallup results. It might take very little to shift vote preference but, in this case, what factors triggered the difference?
As such, I am curious about the "differences in question order" that Newport alludes to. It would be helpful to know what questions came just before the primary vote question on both surveys. One clue may be that the new USA Today/Gallup poll applies their standard "likely voter" model to the general election results. Gallup's model involves eight questions about past voting, vote intent, political interest and knowledge of voting procedures. Did any of those precede the primary vote preference question?
UPDATE - Gallup's Frank Newport clarifies: The Gallup Daily screen is a bit different from the procedure used for the USA Today/Gallup poll. Details -- and more information on the questions that came first on the USAT/Gallup poll -- now posted here.
Obama 46, Clinton 43... McCain 53, Huckabee 27
Clinton 48, McCain 43... Obama 51, McCain 41
CNN/Opinion Research Corporation
McCain 50, Clinton 46... McCain 48, Obama 47 (LV)
McCain 49, Clinton 47... Obama 49, McCain 45 (RV)
Note: The USA Today/Gallup survey is a different sample than the Gallup Daily Tracking survey, which shows a much closer margin.
Public Policy Polling (D)
American Research Group this morning released results for new surveys for Ohio and Texas, and last week for Vermont and Rhode Island (and yes, we overlooked the RI & VT surveys last week - apologies). Each survey is based on 600 likely voters for each party primary:
Rhode Island (2/20-21):
(Left over from last week due to a weekend spent at a family gathering...)
Chris Ison and Rob Daves look closely at the dearth of pre-caucus polling in Minnesota and the state of pre-election surveys generally -- well worth a click.
John Diaz speculates on "why the polls are so wrong."
Kathy Frankovic explores lessons learned about how polls are "being used and reported this year," including qualms about aggregating polls as we do here.
Jennifer Agiesta digs into the exit polls to examine subgroup variation in Democrats satisfaction with their candidates.
Mark Mellman (writing before the Wisconsin primary) sees potential for Barack Obama in the open primaries in Wisconsin, Ohio and Texas "even while the basic demography may favor Clinton."
Jay Cost sees evidence of momentum in the Democratic race.
Brian Schaffner uses poll averages to project March 4th delegates
Carl Bialik computes the odds of Clinton's "lucky break" in Alabama.