Pollster.com

Articles and Analysis

 

Rasmussen's State Level Party ID Weighting

Topics: Party Weighing

I received an email reply from Scott Rasmussen to the questions I posted on Monday about their party identification numbers for June and about their procedures for weighting by party at the state level.

First, I had noticed a very slight difference (0.13%) in the Democratic party identification advantage they reported at the national level for June (9.37% vs 9.5%). Rasmussen confirms my hunch that filtering for "likely voters" narrows the advantage ever so slightly.

Second, he answered a far more important questions, how do they set targets for their party ID weights at the state level:

The question of states is more challenging and one we continue to work. Our initial targets are set by playing off the national numbers. We note changes from 2004 and/or 2006 and make comparable changes to the state targets from our polling in those years. Broadly speaking, if the number of Democrats are up 5 nationally compared to an earlier period, then the state numbers would be up five too. Due to demographic differences, not every state moves completely in synch with the national numbers, but they are close in our targeting formula.

Then, we monitor the state by state results as we conduct state polls and are in the process of making some modest mid-year adjustments now (in most states, we have at least 3,000 state specific political interviews to draw from, plus our national political tracking data, plus our baseline numbers from the other poll). Realistically, though, the current adjustments are very small. To this point, the national shifts appear to provide a good indicator. As we head to the fall, we will poll every competitive state at least weekly and do larger samples. This will enable our dynamic weighting process to draw upon up to 10,000 state-specific interviews or more to set the targets for key states.

Thoughts anyone?

 

Comments
avatarMM:

I dunno... polls are fun, but i take rasmussen's with a grain of salt.. hard not to when the guy predicted GOP dominance for an entire generation.

also i have little respect for survey USA.. their numbers are very odd. in Iowa they had McCain beating obama among blacks.. their polling size is probably so small that those types of errors will happen (granted Iowa has very few blacks but come on)

and these are the 2 polling firms that seem to put out the most polls

____________________

avatarMM:

I dunno... polls are fun, but i take rasmussen's with a grain of salt.. hard not to when the guy predicted GOP dominance for an entire generation.

also i have little respect for survey USA.. their numbers are very odd. in Iowa they had McCain beating obama among blacks.. their polling size is probably so small that those types of errors will happen (granted Iowa has very few blacks but come on)

and these are the 2 polling firms that seem to put out the most polls

____________________

Allen:

It seems to me this sort of procedure is trading off poll-to-poll variability for a potential bias in the results, i.e., variability goes down but the potential for systematic error increases. I don't mind variability because you can see it in the results and it encourages caution. Bias on the other hand remains hidden until election day and may lead you to make a faulty conclusion.

Does Rasmussen also publish "raw" numbers, adjusted only if necessary to achieve a random sample, or numbers that have only been weighted by attributes that cannot significantly change during the course of an election (gender, age, ethnicity, education, etc.)?

____________________

phillybooster:

It seems to me that there has to be some "reversion to the mean" assumption is changes in voter ID levels.

There may be a 5% change in party ID nationally, but Massachusetts probably less so (because everyone was already a Democrat anyway) and Utah probably more so (but not nearly enough to swing an election).

Overall, the differences wouldn't matter because it would be closest to 5% in the swing states, and therefore you would be using the right number in the cases for using the right number is most important, but just because you are getting the right answer doesn't mean you're doing it the right way.

____________________

jsh1120:

Mark,

Thanks so much for following up on this issue. I have a little trouble understanding what Rasmussen is saying at points, but the bottom line appears to be that a baseline partisan split based on 2004 or 2006 (I assume US Senate or Governor) polls is simply "adjusted" by the national change percentage. All else appears to be marginal and minor.

I suppose no other approach has more to recommend it given the data available. There are undoubtedly problems in some states as "phillybooster" notes, but I can't think of a better way to do it.

Of course, this approach does raise the old "guy drowns in a river with an average depth of two feet" problem. For example, I was struck by Rasmussen's most recent poll in New Jersey. What appear to be odd results (to me, at least) might come about from applying a weighting scheme in a state with a large number of "moderate" Republicans who (in 2008) are reluctant to admit their partisanship. If that's the case, Rasmussen may be overweighting the prevalence of conservative Republicans in their results.

I continue to be skeptical of the whole idea of partyID weighting, especially in an environment where one party suffers disproportionately from a "brand image" problem. And I continue to be curious about Rasmussen's apparent tendency to find more Republicans than almost any other pollster. But I have to admit that he is being far more open about his firm's methodology this year. And he deserves kudos for that.

____________________

Undecided:

Bottom line... the choice is weighting the polls for political ideology or true random sampling where you get whoever answers the telephone.

As explained, the only way to know the reality is to do numerous polls (samples) and adjust accordingly.

____________________

brambster:

Mark,

This was very useful to know (the state level stuff). I appreciate you chasing that information down.

I think it is this may provide some insight about why in certain states Rasmussen remains a regular outlier. It would seem to me that in Florida, Ohio and Wisconsin, the change in party ID may have surpassed that of the nation as a whole. New Hampshire comes to mind as an example of this sort of switch which started happening in 2004 (before the rest of the country). Then in 2006 we started to see this sort of change in states like Virginia, North Carolina, Indiana and Ohio (i.e. enough to swing seats, but not of course unique to just them).

New Jersey may be a different animal just because they are notoriously out of love with politicians and they seemingly turn out strong for Democrats on the day that matters. I don't necessarily think that NJ is an outlier, they are just a special case.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR