Pollster.com

Articles and Analysis

 

Party ID Wars 2008

Topics: Barack Obama , John McCain , LA Times , LA Times/Bloomberg , National Journal , Party Weighing

My NationalJournal.com column, on the debate about the new LA Times Bloomberg poll and its composition in terms of party identification, is now posted.

On MSNBC's Hardball yesterday, my friend Chuck Todd made a point similar one I made in the column:

This is how [the LA Times/Bloomberg poll] matches other polls. Republican voters less inclined to call themselves Republicans. For instance, John McCain is down 12 points among independents. Why? Because more Republican voters don't want to say they're Republican, so they're saying they're independent. And more Democratic leaning independents and saying, "hey, I don't mind being called a Democrat right now."

So you're looking at that and you say, this is just an enthusiasm gap. Maybe [the Obama margin] is not 12, maybe it's 8, maybe it's not 15, but it's a lead.

The quoted section comes at the beginning of the clip. The whole segment is worth watching for Peter Hart's commentary both about the focus group he conducted in York Pennsylvania earlier this week, and his thoughts about polls and polling generally.

Update: I wrote extensively about the issue of weighting by party identification during the 2004 campaign, and virtually all of it is as relevant today as it was then. For those with questions, start with the first post below, and keep reading:

 

Comments

Obama led by 8 (44-36) among Independents in the LAT/Bloomberg poll ... and that's a stronger performance than it sounds, because the Indie pool is overstocked with habitual Republican voters who are shying away from the damaged brand.

The LAT poll story also states an exceptionally low measure (11%) of Clinton supporters for McCain (contemporary polls yield 32-47% total defections to McCain/Others/Unks). Hard to evaluate further, as the underlying data on this score is unpublished.

____________________

brambster:

@RonK

That would definitely be skewed from others. Even Newsweek showed 18% of Clinton voters defecting to McCain with another 13% undecided, while showing a 15 point overall Obama margin.

The problem however with dismissing the entire poll on just this one thing is the fact that Clinton supporters probably represents less than 20% of the total sample of this poll (no more than 250 respondants), and when the sample gets to that size on a sub-question, the margin of error goes way up.

Part of the skew may have been from including unregistered people along with registered. Many other polls are using likely voters. I recall hearing Chuck Todd saying that he felt that registered voters are a better measure in this cycle since it is a change election and Obama (and Clinton) have energized record numbers of voters in the primaries that could carry through to the general. Nevertheless, it seems to me that including non-registered people in a poll will generally skew it even more to the Democrat as these people tend to be more likely urban, less educated, lower earning people who when they vote generally favor dems.

The gist of this is that their data is likely a reasonably accurate sample, but taking that sample and turning it into a representation of actual vote in November is what is at issue. I suspect that some pollsters are going too far in adjusting data, and some aren't going far enough. That's why the poll averages and trends are much more reliable than individual polls.

____________________

JS:

Mark

Two questions:

1. Thank you for you excellent analysis of weighting. I agree that weighting by a subjective variable like Party ID is dicey. But doesn't weighting by objective demographic variables also introduce another problem. That is, the variable "who would you vote for" is correlated with various demographics, but those correlations are imperfect and have their own sampling distribution, which is non-zero. Therefor, doesn't weighting by variables that have their own sampling error relative to the vote preference variable throw off the sample error theory in evaluating vote preference. Put another way, isn't is possible to have a vote preference result that is exactly at the mean of the sampling distribution (i.e., "spot on") but still have demographics which are statistically divergent from their population means, because they are correlates with r 0.

This is not an argument against weighting, but merely this question: Aren't you in to statistical trade offs (in terms of pure sampling theory)?

If so, what is a reasonable way to evaluate.


2. Does this issue, in addition to the even more problematic on of weighing by shifting subjective variables like Party ID, one of the issues that explain some of the differences in polls? If so, will you be giving any guidance on that as key polls come out. That would be great.

Thanks.

JS


PS To even further complicate the subject, weighting by objective variables are also subject to measurement error. I recall a panel study done by U Mich's SRC in which gender over time had a correlation surprisingly

____________________

jsh1120:

Mark,

Thanks so much for reposting your articles on weighting from 2004. Having received my training at the UofMich's SRC back in the day when Party ID was seen as a more or less immutable attribute (much like religious identification)and as an almost certain predictor of candiate choice, I remain conflicted about the use of partyID as a weight variable.

It's obvious that partyID is considerably more malleable than it used to be and the extremely negative connotation of even being a "Republican" in 2008 may well differentially influence both response rates (leading to refusal bias) and to an apparent underrepresentation for the GOP in polls that don't weight by partyID.

At the same time, I recoil at the practice of weighting samples by a variable that (a) cannot be independently validated and (b) is so highly correlated with candidate choice that it's nearly a surrogate for the dependent variable (candidate choice) that one is trying to measure. (If one believes the "true" candidate split is 60/40 for Obama, for example, why not simply weight the sample to match that split and be done with it?

All in all, I suspect that the 2008 election may present a "perfect storm" of challenges for pollsters. With a damaged GOP "brand," I suspect it may be difficult to find respondents willing to admit their GOP identification. With a huge enthusiasm gap between the parties, I suspect that traditional "likely voter" models may be problematic (and likely to overestimate GOP turnout based on past behavior.) Add to that the potential of the Obama campaign to motivate new (and occasional) voters and the same models may underestimate Democratic turnout.

Finally, we don't know, of course, how well pollsters would have called the 1932 "realigning" election. But if 2008 turns out to be another one, the guidance provided by partyID in predicting voting behavior may well be problematic.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR