Pollster.com

Articles and Analysis

 

Party ID: The Case for Weights and Historical Margins


[Editor's Note: We are pleased to add yet another contributor to the Pollster.com lineup. Kristen Soltis is currently the Director of Policy Research for The Winston Group, a Republican affiliated public opinion research and strategic consulting firm in Washington, D.C. Welcome Kristen!]

The debate over party ID and whether or not weighting for party ID is appropriate has raged on for years, with a very thorough treatment by Mark Blumenthal and others that raises good questions about whether or not party ID is stable at the individual level. Recent media polls with wide ranging spreads between Republicans and Democrats make it all the more appropriate to bring this debate back.

Those on the side favoring weighting say that it is important to compare "apples to apples", to see if more people actually are voting for Obama than last month, or if we just happened to get a sample more favorable to him. On the other side, you have folks who view partisan identification as a question response, not a demographic group, and view weighting by party as methodologically unsound.

Though it's controversial, I believe that weighting for party ID is appropriate if done in a manner consistent with historical norms. I fall into the camp that believes party ID is far more static - that voters can change their preferences and the intensity of their partisanship often, but do not as frequently take the step of giving themselves a new party with which to identify. To me, party ID falls somewhere in between "demographic fact" and "variable question response". Preventing wildly fluctuating data outside historical norms provides a better picture of what real movement is occurring in the electorate on questions like the ballot test.

On Election Day, the partisan makeup of the electorate is rarely dramatically different from the election four years prior, and the exit polls from the last twenty years corroborate this. The National Election Study at the University of Michigan back in the 1960s showed party ID was stable at the individual level, but some have dismissed this as an example that works today. So let's take a look at more modern day politics, with a time frame of last twenty years (presidential elections since 1988). Washingtonpost.com has a great, simple table of this exit poll data.

In 1988, Democrats had a three-point party ID advantage over Republicans (38-35). In 1992, Democrats still had a three-point party ID advantage over Republicans (38-35). In 1996, that advantage increased to four - a shift of one point (39-35). In 2000, Democrats were steady, up by four (39-35), and in 2004 they dropped to even (37-37).

During presidential years, over the last five presidential elections, the biggest party ID gap was four points, and the greatest swing was four points as well.

Arguments can certainly be made that in this environment, Democrats should be expected to have a huge partisan shift in their favor. But note that in 2006, when Democrats clearly found enormous success at the ballot box, that the advantage in party ID was only three points (38-35). Polls leading up to the election showed party ID gaps as big as eleven points (Newsweek's poll on Oct 5-6, 2006), rarely showing party ID gaps of less than +5 for the Democrats.

On Election Day, as measured by the exit polls, the party ID divide was just three points.

Just because people are voting Democratic doesn't mean they are becoming Democrats.

Truth be told, the decision to use weights for party ID has everything to do with whether or not a pollster views party ID as a "response" or a "demographic", and when it is a fairly stable characteristic of the electorate, I feel comfortable placing it on the spectrum closer to "demographic". It's not perfect, to be sure, but I'd rather compare surveys month to month and observe movement by comparing apples to apples.

However, whether or not weighting is used, the partisan makeup of a poll must factor into the understanding of whether the poll is presenting a realistic piece of information. I certainly don't believe all polls must weight for party ID in order to be useful. But regardless of whether the party ID is organic or weighted, it should still look reasonable.

So let's take a current example that I have trouble with. As "bambi" noted in the comments (taking quite a bit of heat, and with some calculations that I do disagree with) just this morning about the most recent CBS poll, after weighting for demographics, the difference between Republicans and Democrats nearly doubles. While the unweighted sample has 317 Republicans and 381 Democrats (out of 1034 adults), the weighted sample has 284 Republicans and 406 Democrats. This changes the spread from a 6 point spread (31-37) to a 12 point spread (27-39).
Truth be told, if a poll shows a six-point party ID spread, I wouldn't immediately dismiss it. Furthermore, the CBS poll is of adults, not registered or likely voters, so that gives it freedom in my opinion to veer a bit outside the norms. I'm not dogmatically tied to historical precedent, though I think it's very instructive in determining what is "reasonable".

But a twelve point spread? Whether this is a blip or what consistently turns up in the numbers, I have incredible difficulty believing that a margin of that magnitude is an accurate reflection of the electorate. A six-point lead is within the realm of possibility given a really great year for Democrats. But a twelve-point spread is simply outside the bounds of history, given that in twenty years of political change and history, the greatest margin has been four.

 

Comments

In my comments quoted in the NY Times story on the drop in GOP registrations a couple of days ago, I tried to put this into historical context. If the polls measuring self-identified partisanship shifting towards the Democrats, the exceptional Democratic primary voter turnout, and the pro-Democratic shift in voter registration are all part of the same trend, then we may be in the midst of a rare so-called "realigning election," where the party support bases fundamentially shift. The last such realigning election occured in the election of 1932, before the advent of modern polling. We simply do not know what a realigning election actually looks like when it is taking place, or if such mythical creatures actually exist. But if we are in a realigning election, perhaps the 12-point spread on party id is accurate, despite recent history.

____________________

1magine:

Michael - Agreed. I for one would not define Presidential history by the last 20 years; at a minimal 40 years of voting history and party affiliation might be useful.

____________________

Alan Abramowitz:

The Democratic advantage in party id is unlikely to be as large as 12 points on Election Day but may well be larger than 3 or 4. There has clearly been a substantial increase in Democratic identification in the voting age population over the past five years with the largest gains among younger voters. Many polling organizations have documented this. I would be surprised if that is not reflected in the composition of the actual voters.

____________________

brambster:

Party ID is such a strong indicator of voter preference that if you first poll for party ID, and then you weight subsequent polls based on that party ID, you are multiplying errors between two polls and causing a much greater MoE than the raw sample suggests. I think it's clear that Rasmussen suffers from this, and the CBS poll referenced in this story too clearly suffered from this. It becomes much more than so-called "noise", it causes inherent unreliability, especially with polls of just 500 LV's.

I also agree with Michael that this could well be a realigning election. I certainly believe it is at least, confirming a trend that started in 2006. The major factors supporting this also should include the generic ballot preference, confirmation of the 2006 election of these numbers, increased registrations, the well publicized Democratic primary. Obama does get some credit for this, and he is a different type of candidate from what we are used to seeing, but this shift was first caused by Republican corruption (most House members/districts that lost in 2006 had some sort of scandal), and Bush's incredible unpopularity coupled by very strong Republican party unity (unlike Nixon and Carter).

It used to be that majorities of people looked to Republicans when they were unhappy with or distrustful of government, and it is at least shifting back strongly to Democrats if not favoring them. Even Democrats picked Obama over Hillary largely because they wanted something new and different from the last couple of decades.

____________________

Lompe:

Take a look at the three recent CBS-polls on paty ID:

1: May/June (registered)
unweighted: R:27 D:40; I:33 =13+
weighted: R:26 D:41; I:33 =15+

2: 7.-14.July (all)
unweighted: R:25 D:42; I:33 =17+
weighted: R:26 D:37; I:37 =11+

3:July/Aug (all)
unweighted: R:31 D:37; I:33 =6+
weighted: R:28 D:39; I:33 =11+

Seems like they are weighting on a 3 months basis.
The funny thing is that, without weighting, Obama would have lead by 10% in July and 3% in Aug. But by weighting for party ID it is steady at 6%. Which seems more realistic, but not nessesarily just as exciting...

____________________

In Wisconsin, where the presidential elections in 2000 and 2004 were extremely close, the Party ID edge for Democrats has gone up substantially this year. Pollster.com's very own Charles Franklin has documented the robustness of this trend through multiple polls:

http://politicalarithmetik.blogspot.com/2008/06/trends-in-party-identification-in.html

To the extent that an appreciable number of other states are showing similar trends, that would presumably raise the Democrats' national Party ID edge above the typical 3-4%.

If some movement is indeed afoot this year in Party ID trends, then using previous years' exit polls as a template for weighting this year's polls might not be the best solution.

Rather, the idea of "dynamic weighting," used by Rasmussen, might be better. Dynamic weighting involves averaging the Party ID proportions from the last several polls one has taken, and then using this average to weight one's next poll.

This way, if a pollster obtains a sample whose Party ID composition seems discrepant from other polls, those Party ID numbers are still treated as valid. Yet, by averaging those numbers with those from previous polls to derive the new weights, poll-to-poll volatility is dampened.

I invite everyone to visit my Party ID/Sample Weighting website, where these and other issues are addressed. Just click on my name below...

____________________

Make that, click on my name ABOVE (in "Preview" mode, one's name appears below the posting, but in the final version, it appears above).

____________________

jsh1120:

Ms. Soltis appears, at least to my reading, to make a crucial error in comparing party id to other "demographic" variables used for weighting. In the case of attributes such as gender, age, education, and (even) income, there is a wealth of independent measures that provide a basis for weighting. In the case of party id, we have only other surveys, almost all of which are measured in previous elections, to provide an independent source for weights.

(It's true, of course, that other demographics may be based on survey data, but I don't think anyone would argue that census data, for example, is as subject to measurement error as much smaller samples from previous telephone surveys or exit polls.)

In short, I can't get around what I think is the central problem here: We simply don't know what the appropriate partisan identification of the current population is. Add to that the fact that as an attitudinal variable it may be changed by a respondent while other demographics cannot and the issue of whether it is stable or not in the aggregate seems irrelevant in terms of treating it as a WEIGHTING factor.


____________________

Nick Panagakis:

I don’t weight by party ID. In Illinois where I have 20+ years of historical data, party composition changes some from poll to poll in the months leading up to election day. Not an issue. I have some questions though.

1. There are various forms in use for asking party ID. Do pollsters who weight by party ask the same question as used in exit polls? Their question reads: “No matter how you voted, do you usually think of yourself as a” (and provides these answer choices) “a Democrat, a Republican, Independent, or something else”. Seems like their question should be identical. Are they?

2. Not questioning the exit polls or data posted by the Washington Post. But the consistency of party ID is surprising given differences in voter turnout over recent presidential years.

Turnout from the FEC: 2004, 122 million: 2000, 105 million; 1996, 96 million; 1992, 104 million; and 1988, 92 million.

Yet throughout this period, Republicans remained at 35% from 1988 through 2000 then went to 37% in 2004. And Democrats were at 38% in ’88 and ’92, 39% in ’96 and ’00, and 37% in ’04.

Can anyone explain such consistent party composition while turnout ranged from 92 million to 122 million? Any thoughts or ideas? I find this puzzling.
.

Nick

____________________

brambster:

Nick, Rasmussen just yesterday in a national tracking poll asked me "Press 1 if you are you a Republican, press 2 if you are a Democrat, press 3 if you are independent or affiliated with another party". My wording on the third choice might be a bit off, but I did take note that they didn't ask if I "identified more" with a party, but rather if I was of a party. I am very liberal, but I also refuse to register with a party so the way it was asked, I identified as independent, but if I was asked whom I identified more with, I would have chosen Democrat.

Rasmussen seems to ask certain common questions in their own way such as presidential approval with 5 choices instead of 3, and I believe that I have seen that other pollsters have asked about identification with a party rather than Rasmussen's stricter "are you" way.

I am not positive about this, but I believe that the exit polls ask about party registration which is even stricter. Party registration is of course a known quantity in most states and is more static than what is commonly referred to as Party ID which is what the polls use for weighting.

I'm no expert, but I hope this helps.

____________________

Texas Blue Dog:

The inherent difficulty in the partisan analysis (and Brambster's situation alludes to this) is that 'independents' are so numerous that they effectively constitute a third party. I myself am another example of the Rasmussen error that Brambster notes. I identify MUCH more closely with the Democratic Party, but I have a natural Libertarian lean in my political thinking and differ with the Democrats on a few major issues.

As to why turnout has differed widely over the last several elections, I think the independent faction can account for much of this. And it is important to note that 37-31 in favor of Democrats are percentages, not hard numbers. Taking the 37-31 number, that leaves 32% independent. Assuming that Obama and McCain have similar levels of party unity, crossover votes would effectively cancel out, with McCain getting nominally more votes from the exchange. This is why the 32% independents will decide this election.

Given that Obama has a partisan advantage going in (let's say 36-31), he would only need 48-49% of those independent votes to win. He might squeak by with as few as 45%. If he got 50% independents, Obama would win comfortably.

Hence, polls that have a large number of undecideds, weighted or not, are of limited value at best. A good measure of the state of the race might be which candidate would a proclaimed independent favor (Obama has the lead in most measurements of this).

____________________

Gary Kilbride:

Party ID is overblown. The more relevant factor is percentage of self-identified liberals and conservatives. I've charted that and it's clear cut. People may be reluctant to align with the Republican party right now, but it doesn't necessarily mean they don't think of themselves as conservative. There are many instances where Party ID percentages changed significantly but the self-identified percentages remained the same. And so did the voting pattern. Democrats do not make a lasting dent in the terrain unless the liberal/conservative percentages tighten nationwide.

The self-identified numbers reveal how a state is likely to vote, and whether it is changing in fundamental terms. For example, Virginia according to the '04 exit poll was 17% liberal, 38% conservative. That's anything but a swing state. But in '06, Virginia checked in at 21% liberal, 35% conservative. That puts it just slightly red compared to the nation as a whole. A swing state normally is in the 20-22% liberal, 32-34% conservative range.

I'll be extremely interested in those numbers this year, specifically in states like Virginia, Colorado and Ohio. It's also why I'm skeptical about Obama's opportunity in Indiana. That state was 14% liberal, 42% conservative in '04.

Anyway, my projection this year has been for Democrats to be +4, or possibly +5, in Party ID. I'm a believer in long term numbers, not to over react and expect an extreme.

____________________

Nick Panagakis:

Response to “brambster” above.

After checking Blumenthal’s extensive analysis of this subject four years ago (see links in Kristen’s first paragraph), I found that few pollsters, perhaps only one pollster, uses exit poll party ID data to weight their pre-election polls. Consequently, no need for most pre-election pollsters to match exit poll party ID question wording as I suggested. (BTW. My post above includes the party ID question exit pollsters ask and it is not party registration.)

Re: Party Registration. Not a useful measure because only about 20 states have party registration which means you must register as a Dem or GOP to vote in the state’s primary. This is called a closed primary. Other states have open or modified open primaries (including Illinois, recently listed as closed in an electionline.org publication).

Here is an example from a few years ago. We conducted a general election poll in New Jersey, a party registration state that like many states that have open primaries allowing independents to vote in the primary of their choice. Party ID in the poll showed 39% Dem, 28% GOP, and 32% independent. But state records that year show 25% registered as Dems, 19% registered as GOPs, and 56% registered as independents. I think this means many New Jersey voters, as do voters in other states including New Hampshire, like to keep their options open come primary election day so they can vote in the party primary of their choice. Conclusion: party registration is not a good measure party preference.

Nick

____________________

brambster:

Nick, thanks for the correction.

I took note of one of Mark's 2006 posts regarding party ID. While maybe no other active pollsters are using party ID weighting, they mostly track it in some way. The variability in how they ask the questions does cause trouble in comparing the results, but it might be useful to know as the first graph on this post demonstrates:

http://www.mysterypollster.com/main/2006/04/rasmussen_and_p.html

Rasmussen's party ID weighting is likely behind their extreme concentration, and this also shows a trend of Gallup in finding more Republican identifiers than the average. A lot of discussion has been had about how the national tracking polls of both are under-polling Obama in comparison to almost all other national polls. I would love to see a graph showing the difference in party ID for national polls. It might also be interesting to take all polls and generate a mean difference and then adjust the results for that mean difference and see how much different that might be from the polling trends over time (line chart to line chart). This may give a much flatter chart. IMO, there has been almost nothing in the last 2 months that would have caused a real swing in voting outside of the initial Obama bump after Hillary, and a small bump after his trip, and my guess is that the undecideds or sing voters will mostly not start paying attention until the conventions at least.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR