Pollster.com

February 16, 2008

Pre-President's Day "Outliers"

By Mark Blumenthal

The Hartford Courant's Joann Klimkiewicz examines the problems of polling in 2008.

Kathy Frankovic shares her skepticism over polls to tells us which is most electable in 2008.

Frank Newport finds that John McCains "displeases" many conservative Republicans.

Gary Langer says race has been the "single most powerful demographic in vote choices" in the Democratic primaries so far.

David Hill sees evidence that "immigration is a dud as an electoral issue."

Mark Mellman considers the complexities of the "politics of identity" on the Democratic primaries of 2008.

Tom Webster crunches the exit poll numbers on Republicans in Virginia and Maryland that listen to talk radio.

Josh Goodman compiles the exit poll results on abortion and immigration.

Carl Bialik calculates the odds of a tie in Syracuse.

Karl Rove does poll analysis on a white board.

By Mark Blumenthal on February 16, 2008 10:57 AM | Permalink | Comments (13)

TPM Catches a Milestone

By Mark Blumenthal

Talking Points Memo has a headline this morning pointing out something we were too busy to blog yesterday: For the first time, Barack Obama's number on the our national trend estimate (47.1%) is now greater than Hillary Clinton's (46.0%).

We are doing a bit of an overhaul on our database to facilitate inclusion of the daily tracking from both Gallup and Rasmussen Reports. The chart does not yet include data from Rasmussen, largely out of a concern that their more frequent updates would dominate the trend average. We are hoping to revise the chart to include the Rasmussen data soon.

By Mark Blumenthal on February 16, 2008 10:44 AM | Permalink | Comments (14)

The Problems of Primary Polling

By Mark Blumenthal

I have written a lot this year about the surprising level of indecision about the presidential race among many Democrats on the eve of the primary elections, and the problems that uncertainty creates for polling. Jay Cost weighs in this week with an essay that approaches the same issue from a slightly different perspective. His take is especially relevant now that the primaries have moved beyond hotly contested states like Iowa and New Hampshire to states where information levels about the candidates are significantly lower.

Cost reminds us that "average voters do not pay much attention to politics" and that in general elections, partisan identification serves as the "cognitive heuristic" or "mental shortcut" that facilitates decision making despite low information. He points out that party identification is an "incredibly precise predictor of vote choice" and that it makes for stability in poll measurements:

Accordingly, we will see the polls vary only a little bit throughout the campaign. Oftentimes, they will break in late October or even early November. However, the magnitude of the break will be relatively modest.

However, as Cost points out, primary elections are different:

In a primary campaign, voters must choose among candidates who are all of the same party. Partisanship therefore does not enter into their decisions. It is a non-factor. I think this might be inducing the wild swings in the polls. The polls are varying because the voters are; the voters are varying because their partisanship is not stabilizing their preferences. [...]

It thus should be unsurprising that candidate personalities are so influential in voters' decision-making processes. How else do you make determinations when party distinctions are non-existent? Candidates often try to create clear contrasts, but these usually amount to making mountains out of molehills. The average voter is not really paying much attention, anyway. Thus, they have to go by their personal evaluations of the candidates.

And when those personal evaluations are mostly positive, some voters are having a hard time making up their minds. So their choices, as described to pollsters, may be tenuous. Cost's essay is good; go read it all.

By Mark Blumenthal on February 16, 2008 9:49 AM | Permalink | Comments (5)

DC-AAPOR event on Wednesday

By Mark Blumenthal

An announcement for those in the DC area. I will be participating in a discussion on "Politics and Polling" next Wednesday afternoon hosted by the DC Chapter of the American Association for Public Opinion Research (DC-AAPOR). Hope you can join us.

By Mark Blumenthal on February 16, 2008 9:22 AM | Permalink | Comments (1)

Why So Much Volatility in Texas?

By Mark Blumenthal

Not surprisingly, the three new Texas polls we posted yesterday provoked quite a bit of discussion. We have three polls showing very different results for the Democrats, but much more consistency for the Republicans. How can that be?

First, a quick summary: A survey sponsored by the Texas Credit Union League and conducted by two campaign pollsters, Hamilton Campaigns (D) and Public Opinion Strategies (R) has Clinton leading Obama by eight points (49% to 41%). A new automated survey from Rasmussen Reports has Clinton leading by sixteen (54% to 38%) and a new survey from American Research Group (ARG) shows Obama leading by six (48% to 42%). The Republican results are far more consistent, showing John McCain leading Mike Huckabee by margins of four to eight points.

One likely reason for much of the apparent "volatility" in the Democratic results is that the Obama-Clinton vote preference shows large variation on five critical variables: race and ethnicity, gender, age, socio-economic status and party affiliation (percent non-Democratic on party ID). Small changes in pollster methods (such as whether they sample from a list, how they select respondents within each sampled household, what time of day they call, whether they use live interviewers or an automated methodology and how they weight their data) can produce important differences in sample composition that will in turn affect the vote preference results.

Here is the data available online from the three most recent surveys (some of which was posted by our readers in comments yesterday):

Unfortunately, only the TCUL/Hamilton/POS poll provides complete information on its sample composition, although the ARG summary provides percentages for selected subgroups. From these data we can see that the we can see that the TCUL survey includes slightly more Latino voters and slightly fewer African-American voters than the ARG survey. That explains a few points of the difference between them but (as noted below) not all.

The table above also includes sample composition statistics from the 2004 Texas Democratic exit poll, although the 2008 composition will likely be different. Just how different we will not know until the votes are cast, but the exit polls so far this year in other states provide some guidance. The Washington Post's Cohen and Agiesta have put up a very helpful compilation showing the demographic shifts from 2004 to 2008 in 17 states that have held primaries or caucuses so far this year. Women have made up a slightly greater share of Democratic electorates almost everywhere (averaging about a 4 percentage point gain). The percentage of 18 to 29 year olds has also increased in just about every state, up 4 points on average.

The changes in race and ethnicity have been less consistent. Most relevant to Texas are California and Arizona, the two states with the largest Latino populations. In California, the Latino contribution surged (+14), while the African American percentage was roughly constant (-1). In Arizona, the African American percentage as up far more (+6) than the Latino contribution. Cohen and Agiesta also note that black percentage of the Democratic electorate is down slightly in two states (Florida and Virginia) where the Latino percentage increased.

The racial and ethnic composition of the three most recent surveys does not explain the their different Obama-Clinton results. As the following table shows, the biggest difference among the three is that the ARG survey reports an even race among Texas Latino Democrats while the Hamilton/POS and ARG surveys give Clinton a roughly two-to-one lead, comparable to her showing in other states with large Hispanic populations.

Another factor in the "volatility" of these polls -- a factor that is next to impossible to evaluate from the data available -- is how tightly (and accurately) they screen to identify "likely voters." In 2004, the Texas Democratic primary attracted 839,231 voters, 6% of all eligible adults and 5% of all adults in the state. Democratic turnout has increased everywhere this year, nearly doubling on average in primary states (as a percentage of eligible adults) although the state-by-state patterns have varied widely. Texas is all but certain to see a big turnout boost, but just how big is anyone's guess.

They key point here is that polls may yield different results depending on how broadly or narrowly they conceive of the Texas primary electorate. Unfortunately, the degree to which they screen for "likely voters" is hidden from our view.

By Mark Blumenthal on February 16, 2008 9:15 AM | Permalink | Comments (29)

February 15, 2008

Exit Poll Data: Education and Race

By Mark Blumenthal

The Washington Post's Jon Cohen has posted some extremely useful data from the exit polls to his Behind the Numbers blog. He ran the Clinton-Obama vote by education among white voters and found evidence of a large "education gap:"

In each of the states where the Post subscribed to exit polls (and voters were asked about their level of education), Clinton did better among non-college than college-educated white voters. She also outpaced Obama among non-college whites in all 14 of these states, but beat him by more than a single percentage point among college graduates in only five.

This data helps shed some light on the subject of speculation earlier this week, whether Obama is "finally cracking the code of the working class white voter" as one observer put it. Our initial look at the exit polling on this issue was inconclusive, because the official exit poll tabulations show the results by education (and income) among all voters. Since the biggest differences between Clinton and Obama have been by race and ethnicity, the share of African American or Latino voters in each state determines whether they do better or worse among the less-well educated voters in that state.

Cohen's tabulations control for race, showing the percentage by education among white voters in each state, thus allowing for better comparisons across states:

Obama's share of non-college whites in Virginia was, as many assumed, higher than in any other state except Illinois, although his performance among this subgroup has been relatively consistent elsewhere. Obama's percentage of non-college whites in Maryland was similar to most of the other states. Also, as some have speculated elsewhere, his percentage of non-college whites was lowest in three Southern states: Florida, South Carolina and Tennessee.

Somewhat surprising -- to me at least -- is the much larger variation across states among college educated white voters. Obama had large double digit leads among college educated white voters in Virginia, Missouri and Illinois but trailed by double digits among college whites in New York, New Jersey and Florida.

Some of these differences are clearly related to the home state advantages (Illinois, New York and possibly New Jersey). Others may have to do with the relative expenditure of resources (candidate time, television advertising and field organizing) by Obama and Clinton in each state. Do our readers have other theories?

By Mark Blumenthal on February 15, 2008 5:56 PM | Permalink | Comments (18)

POLL: Research 2000 Wisconsin, Gallup Daily Tracking

By Eric Dienstfrey

WISC-TV/Research 2000
(Dems, Reps)

Wisconsin
Obama 47, Clinton 42... McCain 48, Huckabee 32, Paul 7

---
Gallup Poll

National
Obama 47, Clinton 45... McCain 53, Huckabee 28

By Eric Dienstfrey on February 15, 2008 2:28 PM | Permalink | Comments (22) | TrackBacks (0)

Bialik on Poll Mash-Ups

By Mark Blumenthal

Carl Bialik, author of "The Numbers Guy" column for the Wall Street Journal, takes a balanced look today at the pitfalls of something we do here at Pollster, "mashing up surveys from various sources this election year to produce composite numbers meant to smooth out aberrant results." His piece is worth reading in full, as it considers both the benefits and risks of creating composite trends or averages:

Stirring disparate pollsters in one pot has its critics. "That's dangerous." says Michael Traugott, professor at the University of Michigan, and author of a recent guide to election polls. "I don't believe in this technique."

Among the pitfalls: Polls have different sample sizes, yet in the composite, those with more respondents are weighted the same. They are fielded at different times, some before respondents have absorbed the results from other states' primaries. They cover different populations, especially during primaries when turnout is traditionally lower. It's expensive to reach the target number of likely voters, so some pollsters apply looser screens. Also, pollsters apply different weights to adjust for voters they've missed. And wording of questions can differ, which makes it especially tricky to count undecided voters. Even identifying these differences isn't easy, as some of the included polls aren't adequately footnoted.

Bialik quotes both me and Charles Franklin in the column, but here are a few additional thoughts. We do not consider the trend estimates to stand as worthy replacements to the data from individual surveys. The trend lines -- and the estimates derived from their end-points -- are best considered as tools to help make sense of the barrage of often conflicting results from individual surveys. We learned in 2006 that "mashing up" surveys and "smoothing out" the variation between them helps counter the instinct to overreact to variation between individual polls -- some of it clearly aberrant -- that is common in hotly competitive political races. Moreover, while we only plot a few summary measures here such as vote preference and job approval, many of the surveys we report and link to include a wide variety of questions that help illuminate many aspects of public opinion.

Bialik is correct to argue that benefits of averaging lessen when we start to see large and consistent "house effects" separating the results from different pollsters. If a few polls are providing good estimates, while many other polls have misleading results, the mashed up averages may reflect more of the bad than the good. I wrote as much just before the Iowa Caucuses.