Pollster.com: The Demographics of Texas Polls

February 29, 2008

Last fall, I asked all of the organizations that conducted Iowa Caucus surveys to disclose the demographic composition of their samples and several other aspects of their methodology. Although cooperation was mixed, our "disclosure project" demonstrated how polls that are theoretically reporting on the same population of "likely voters" can sample very different kinds of people.

With so much attention focused on Ohio and Texas this week, I thought it would be worthwhile to attempt a less ambitious version. Earlier this week, I sent all the pollsters that had released surveys in either state in recent weeks to disclose some of their demographic composition and to estimate the percentage of Texas adults that their surveys represent.

While new polls have been appearing every day, I want to report the responses so far, starting with Texas.

The demographic mix is especially important in Texas given the large percentages of both African American and Latino voters there. Fortunately, in this case at least, we now have a fairly complete look at how these polls of "likely Democratic primary voters" differ demographically. When the data was not already in the public domain, I received quick cooperation (in Texas) from the pollsters at Washington Post/ABC News, Constituent Dynamics, Hamilton Campaigns and Public Policy Polling (PPP). Also, an encouraging number of pollsters have included demographic profile data their Texas releases, including several that are typically more reticent, including ARG, Rasmussen Reports. And thanks to the Houston Chronicle, even the Zogby/Reuters/C-SPAN poll helped make the world a "better place" by making cross-tabs featuring demographic composition available on Chron.com.

03-01 texas demos4.png

As in Iowa, the results show considerable variation, particularly on the Latino or Hispanic percentage of the samples, which vary from a low of 24-26% (ARG) to a high of 39% (Post/ABC). Other categories also show wide variation including the percentages of African Americans (from 14% to 23%), women (from 51% to 58%) and voters over 65 years of age (from 15% to 26% ~~30%~~; comparisons by age categories are especially difficult, since no two pollsters report exactly the same age breaks [Update: At "Joe's" suggestion, I've added some additional age breaks where available]).

I have included comparable numbers from the 2004 exit poll,* although we will not know what the "right" answer is until the votes are cast and results from this year's exit poll are available.

If they had not yet done so in their public release, I also asked pollsters to estimate the percentage of Texas adults represented by their samples, which is a decent measure of how tightly they screened for likely voters. The percentage of eligible adults that participated in the Texas Democratic presidential primary was just 6% of eligible adults in 2004 and 9% of eligible adults in 2000 (if calculated as a percentage of all adults, including non-citizens, the percentages would be 5% and 8% respectively).

03-01 Texas adults percentage.png

Even though the following table includes results for just four pollsters, the range of adults represented** is huge, from a low of 7-8% for the Texas Credit Union League(TCUL)/Hamilton Campaigns/Public Opinion Strategies poll to a high of 40% on the first poll from SurveyUSA. The TCUL poll obviously comes closest to past turnout, although turnouts have been much higher in other states so far this year than in 2004. The ABC News release concedes that an actual turnout of 24% of adults is "unlikely" but reports that "vote preference results are similar in likely voter models positing much lower turnout."

Next, Ohio... and then after posting all the statistics, I'll come back and speculate about what they might mean for what everyone cares about: where the race stands heading into the final weekend. Given the time crunch, I put these tables together quickly. So if you spot a typo or can help fill in a blank that I've missed please send an email (to questions at pollster dot com).

Footnotes:

*UPDATE: All of the 2004 exit poll results in the table above are from the final weighted data available from the Roper Center archives. Some of the percentages differ slightly from those posted on election night 2004 by CNN and still available online. The difference is likely due to final weighting done after 10:43 p.m. on March 9, 2004, the time the CNN tables were last updated. An earlier version of the table posted above was based, in part, on the CNN results.

**For the ABC/Post, CNN and SurveyUSA polls, we estimate the percentage of adults represented by dividing the number of interviews conducted among likely primary voters by the number of adults interviewed. Since those samples of adults will include non-citizens, and since non-citizens are 12% of the Texas adult population, I calculated a range for the two polls -- PPP and TCUL -- that sample from lists of registered voters (see "Update II" of this post for more explanation).

[Table updated on 3/1 to include Below/WFAA/Public Strategies surveys].

-- Mark Blumenthal

February 29, 2008 in Disclosure, Divergent Polls, The 2008 Race

Comments

sandy:

you know what i wish you'd do and i've been searching high and low... is to compare the surveys on accuracy and reliability against outcomes and show their results going forward on the upcoming races. so we know zogby is very pro obama but he blew it in new hampshire. how was his track record on super tuesday? same for the rest. who gets the highest score? who's got an axe to grind which is certainly transparent when considering the demographics. could you please do this????

Posted on February 29, 2008 6:03 PM

____________________

BitJam:

The variability in the demographic numbers certainly explain some of the great variability we've seen in the poll results. For example, assuming the Latino vote is breaking for Clinton 60/40 and the African-American vote is breaking for Obama 80/20 then the demographic differences between ABC/WaPo and the 2004 exit poll would give a swing of roughly 5.4%.

Posted on February 29, 2008 6:17 PM

____________________

BitJam:

Sandy,

Take a look at the "report card" links on the pollster home page.

Posted on February 29, 2008 6:50 PM

____________________

Anonymous:

Do any of the pollsters give a breakdown on demographics of people who have voted early?

Posted on February 29, 2008 6:58 PM

____________________

Brian Schaffner:

With regard to trying to get at which pollsters have more realistic screens, I wonder if we can get some sense of this from early voting in Texas. According to the Sec. of State's website, at least in the 15 most populated counties, about 9% of registered voters have already voted in the Democratic primary.

http://www.sos.state.tx.us/elections/earlyvoting/2008/feb28demo.shtml

So, we already know that the TCUL/Hamilton poll is off on that measure. Based on the Chronicle tabs you link to, the Reuters/C-Span/Zogby International Poll suggests that about one-third of likely Democratic voters have already voted. If that is true (and two-thirds have yet to vote), then turnout in the Democratic primary would be somewhere in the range of 25-30% of registered voters. That would put the ABC News/Washington Post and PPP polls closest to the mark on that measure.

Of course, these are just some rough back-of-the-napkin calculations. But I do think the early voting can be used to give us a clue of what turnout is eventually going to be.

Posted on February 29, 2008 8:03 PM

____________________

Michael McDonald:

I agree that early voting is a good tool to project election turnout, and have used it with much success.

Here are the percentages of early votes in the following Texas elections:

2000 P: 16.5%
2000 G: 23.9%
2004 P: 17.9%
2004 G: 32.6%

Today (Friday) is the last day of early voting in Texas. In the 15 populous counties reporting early vote totals through Thursday, 959,666 people have voted in both the Democratic and Republican primaries. I believe that looking at both primaries is the more correct approach to forecast turnout since Texas has an open primary (there is no party registration, except that voting in a party's primary affilates the voter with the party for the remainder of the year).

There are many unknowns. Will early voting rates be similar to a typical general election or primary election? Will the 2008 primary reflect the increasing popularity of early voting? Will people planning on participating in the caucus wait until Tuesday to vote? Have the presidential campaigns been mobilizing people to vote early, whereas in previous primaries the campaigns were not active?

Fiddling around with the numbers, if we assume that 1.3 million people voted early (projecting through Friday and factoring in the other non-reporting counties), and that early votes will constitute 25% of all votes, then turnout in both primaries will be 5.2 million or 35% of those eligible (or 29% of VAP). I think this is about right. It is in line with turnout in Wisconsin (which has Election Day registration) and California (which has permanent absentee voting); and it splits the difference between the historical primary and general election early vote percentages. But, see all my unanswered questions above.

Posted on February 29, 2008 9:55 PM

____________________

Joe:

Why do you choose to use 65+ as a breakpoint when clearly 45+ has been the marker in many comparable states? That is what I would like to see...

SUSA predicted that 51% of primary voters will be under age 45, (TalkLeft post) a group that has been consistently for Obama during the primaries. In 2004, that age segment was no more than 33% of primary voters, and has been consistently less than 40% during 2008 in the states from California to Florida, and around 33% in the states immediately surrounding Texas.

Oh, down the memory hole.... SUSA reposted its demographics with different age segments so that it is not directly verifiable with 2004 or 2008 exit polls. Good thing they "fixed" that.

Posted on February 29, 2008 10:30 PM

____________________

Mark Blumenthal:

Joe,

Thanks for the suggestion. Your comment made me realize that I had overlooked the "younger than Obama" subgroup in the SurveyUSA tabs. I crammed the columns together a bit more in the table and added 18-45 and 18-50.

Also, please note that the TCUL poll and the 2004 Exit poll reported results for the 18-44 year old cohort, not 18-45.

Posted on March 1, 2008 9:29 AM

____________________

tom brady:

The latest wfaa belo tracking poll is out and suggests Texas is essentially a dead heat (it has Clinton up by 1%) how do the demographics of that poll match up with these other ones?

Posted on March 1, 2008 11:02 AM

____________________

Joe:

Mark,
Thanks for taking my comment seriously. I was a bit sleep-deprived Friday night and made a couple of mistakes (including the multiple submissions). I thought SUSA had removed the "McCain" crosstab on Friday night... I had the memory hole, not SUSA...

Your table is excellent information. It helps to see what demographic shifts from 2004 to 2008 these models are predicting, some favorable and some unfavorable for each candidate.

Posted on March 3, 2008 5:54 PM

____________________

Ray:

Wait, they really think there won't be any increase in Black turnout?

Posted on March 3, 2008 9:59 PM

____________________

Eric P.:

The composition of the likely electorate may look highly variable from survey to survey but considerable variation is expected due to sampling error. Assuming that the polls do not weight to some predetermined percentage of Hispanics, a series of polls sampling 450 democrats, would be expected to have confidence intervals of +/- 43% (assuming a "true" value of about 29%, the mean of all polls). If the data are weighted, the expected confidence band will be larger -- perhaps a lot larger. Thus we should expect almost all polls to be between 24 and 34 with modest weighting and perhaps from 21 to 37 with highly skewed weights.

That's not far off from the results of the surveys. Uncertainty is an inherent part of the business even when all the polling firms are using best practices and have likely voter screens that are effective in discriminating among those who will and won't turn out.

Let's not ask more of these polls than statistical theory says they are capable of giving!

Posted on March 4, 2008 1:10 PM

____________________