Pollster.com

Articles and Analysis

 

Rasmussen, Massachusetts and Party ID

Topics: Automated polls , Massachusetts , Nate Silver , Party Identification , Party Weighting , Rasmussen , Scott Rasmussen

This certainly seems like a banner week for blogging about pollster Scott Rasmussen, as I count at least three entry-worthy topics on the automated polltaker: (1) the flurry of commentary surrounding the piece by Politico's Alex Isenstadt on attacks from the left on Rasmussen's credibility (2) reporting Monday by the liberal blog Think Progress showing that Rasmussen was paid $140,500 by the 2004 Bush campaign for survey research and the good question that raises about whether sites like ours should label Rasmussen as "[R]" for Republican (3) yesterday's new Rasmussen survey of likely voters in this month's special election in Massachusetts.

As commenting on all three at once exceeds both my time and mental bandwidth, I'm going to start with the third and most timely topic, but I will come back to the other two later this week.

Rasmussen's Massachusetts survey, consisting of 500 automated interviews of Massachusetts likely voters conducted in just one day (Monday), shows Democrat Martha Coakley leading Republican Scott Brown by just nine percentage points (50% to 41%). That's a surprisingly narrow lead in a heavily Democratic state that Barack Obama carried by more than 26 percentage points (61% to 36%). Even in 1994, a banner year for Republicans, Ted Kennedy defeated Mitt Romney in an unusually competitive reelection contest by 17 points (58% to 41%).

Nate Silver looked at the Rasmussen results by party and extrapolated that the survey consisted of 52% Democrats, 21% Republicans and 27% in the independent/other category:

Although there are lots of different ways to ask about party identification, typically that's not what we see in elections in the Bay State, as the number of independents is usually much higher (43 percent of Massachusetts voters were independent/other in 2008, and 51 percent are registered as independents). They're also showing an electorate that is 39 percent liberal, 34 percent conservative, and 27 percent moderate; that compares to 2008 exit poll demographics of 31 percent liberal, 19 percent conservative, and 49 percent moderate.

So Rasmussen's theory on this election, basically, is that the people in the middle won't bother to show up; there are many fewer independents and many fewer moderates in their sample than you usually get in Massachusetts. Instead, it will be a race between the bases.

My first reaction is that while there are indeed different ways to ask about party identification (and even more ways to ask about self-reported ideology), it's a bad idea to compare official party registration statistics (that tally how many voters check the box for Democrat or Republican when they register to vote) with survey questions about party identification (typically: "when it comes to politics, do you think of yourself as a Democrat, a Republican or an independent?"). Depending on the state, the two measures can produce very different sets of numbers.

Back in October, Monmouth University pollster Patrick Murray explained why officially "unaffiliated" voters in New Jersey are very different from the "independents" identified by pollsters. I suspected that Massachusetts, with its very high percentage of non-partisan registrants, might produce similar differences, so I emailed Andrew Smith, director of the University of New Hampshire Survey Center who frequently conducts surveys of Massachusetts for the Boston Globe. Here is his take:

[It's] important to point out that a high percentage of the registered Independents in MA (they're actually called unenrolled) are really either Democrats (36%) or Republicans (34%) when you look at PARTY ID. (We use the Univ of Michigan question and I recode leaners into the partisan buckets). Calling them "Independents" makes it look like there is a large pool of free-thinkers out there up for grabs, which is simply not the case ... not in MA, not in NH (a regular media story during the NH Primary is about the large number of Independents up for grabs, a story which sounds good, but has no basis in fact!), not anywhere!

Those who are registered Unenrolled in MA are less interested in elections and less likely to vote than are registered Republicans or Democrats. This phenomenon is consistent across the US (see "The American Voter Revisited" for the most recent in a long line of studies making this point). It's my sense that the 2010 MA special election will have low turnout and the percentage of voters who are registered as either Democrat or Republican will be higher than the percentage registered as such among all MA adults.

Let me explain that a little more slowly. The Boston Globe/UNH poll routinely asks respondents about both their party registration and their party identification. Early in their interview they ask: "Are you registered to vote as a Democrat, Independent, Republican or something else?" On their September survey, according to data Smith provided, 36% of all registrants said they were registered as Democrats, 14% said they were Republicans and the rest (50%) reported they were unenrolled.

Then at the end of their survey they ask: "GENERALLY SPEAKING, do you usually think of yourself as a Republican, a Democrat, an Independent or what?" To those who initially identify as independent, identify with another party or offer no preference they ask a follow-up: "Do you think of yourself as closer to the Republican or to the Democratic party?"

When Smith combined the initial identifiers with leaners in September, he found 50% were Democrats, 32% were Republicans and only 19% remained independent or without a preference. And a cross-tabulation of the UNH party registration and party identification questions shows that more than two thirds of the unenrolled voters identify or lean to either the Democratic (36%) or Republican (34%) parties.

So unaffiliated/independent was 50% on the party registration question, but only 19% on party identification (with leaners allocated). So again, it is a mistake to expect the party identification results produced by a poll to match the party registration statistics produced by the Secretary of State (and an even bigger mistake to weight results of a poll to match those statistics, but that's a topic for another day).

But wait, the party identification results I cited are for adults. Does the Democratic advantage narrow among "likely voters" in Massachusetts?

Yes, although the results that Smith provided from the September Boston Globe/UNH poll did not ask about the special election. They did ask, however, about interest and likelihood to vote in the general election for governor in November 2010, and thanks for Andrew Smith, I can provide those tabulations below:

2010-01-06_GlobeUNHpartyID.png

As the table shows, if you narrow the survey based on interest in the gubernatorial election, the Democratic advantage narrows considerably, from 17 percentage points among all adults (49% to 32%) to just three percentage points among the 35% of adults that say they are extremely interested (45% to 42%). On the other hand, when the UNH pollsters asked voters if they were likely to vote in the November 2010 general election, the Democratic advantage actually grows slightly, to 19 points (51% to 32%). Which of these, or what combination, might provide the best "model" of a true likely voter? There's no obvious answer -- welcome to the highly varied "art" of likely voter modeling.

But wait...the point of all of this is how these numbers compare to extrapolated party numbers produced by Nate Silver for the Rasmussen poll, and at first blush, the Globe/UNH numbers are not that far off. If anything, Rasmussen's party results (52% Democrat to 27% Republican) are more favorable to the Democrats than the Globe/UNH numbers from September.

I had even considered including the Rasmussen numbers in the table above, but decided against because the comparison is a bit misleading. The problem is that Rasmussen asks a very different partisanship question:

If you are a Republican, press 1. If a Democrat, press 2. If you belong to some other political party, press 3. If you are independent, press 4. If you are not sure, press 5.

Does this question ask about party identification or registration? Given the absence of the "do you consider yourself" clause and the use of "belong to, a respondent might interpret it either way. And I'm assuming that since Rasmussen uses an automated method, the respondent can interrupt the question at any time to choose a selection, something I suspect they tend to do more readily toward the end of the interview (especially if they are feeling impatient to get off the phone).

Also, notice that unlike the Globe/UNH question, Rasmussen does not include a follow-up to press independents on how they lean. Yet if you click on a Rasmussen result on our national party ID chart, you will see that when asked of national adult samples, Rasmussen results tend to produce more partisans (about 12 percentage points more, on average) than other pollsters (roughly 9 points higher on Republicans and 3 points higher on Democrats).

Why is it different? Does the combination of a different question and mode effectively push some independents to say how they lean (with a bigger push toward the GOP)? Or do Rasmussen's sampling and calling procedures yield an adult sample that skews more partisan and Republican, even before they apply their likely voter screen and party weighting? You can make a case for either argument, but I don't have conclusive evidence to resolve this puzzle.

Also, Rasmussen typically weights their statewide pre-election samples by party to targets derived in a somewhat fuzzy process. How did they determine their party weighting targets for the Massachusetts survey? How much did the party weighting alter the results (as compared to weighting on demographics alone)? What percentage of Massachusetts adults passed the screen and qualified as likely voters?

And most important, why aren't answers to these questions disclosed on a routine basis on RasmussenReports.com? Keep in mind that Nate Silver had to extrapolate his estimate of Rasmussen's partisan balance, and even that came from crosstabs available to subscribers only.

I'll have more to say later about the questions of bias, intentional and otherwise, that have been swirling around Rasmussen this week. But until pollsters like Rasmussen start disclosing more about the numbers they produce, it is hard to do much more than speculate about whether poll like the one he did in Massachusetts are as representative as they should be. Is this new poll "horribly, terribly wrong?" With so little information to go on, it's hard to say.

Update: Harry Enten (aka "poughies"**), the Dartmouth student who wrote a guest contribution a month ago on modeling gay marriage referenda, takes issue with my conclusion with an intriguing comment below that argues that the Rasmussen poll "has too many Republicans and not enough independents." He reaches this conclusion by comparing the relationship between results for party ID and actual registration in a previous Massachusetts congressional district race polled by SurveyUSA (which asks a more traditional party identification question). It's worth reading.

**And yes, he gave me permission to reveal his identity.

 

Comments
poughies:

I wish Rasmussen had more disclosure. I think your point about it being "difficult" to really understand what is going on is right close to the truth.

I do disagree with your overall assessment. I think the poll has too many Republicans and not enough independents.

You, yourself, acknowledge that they don't ask the follow up.

I thought Massachusetts' Special 5th Congressional District election of 2007 was a perfect test case to test the hypothesis that "indies" won't show up. Both are special elections and have received little press coverage. The 5th is slightly more Republican, but not overly so. Both have similar demographics.
While data from the Secretary of State are somewhat sparse, a series of polls by SurveyUSA (another automated firm) were conducted during the leadup to the 07 special election. Both of the polls nailed Tongas' total and underestimated Ogonowski's total by 3-4 points (easily accountable by an overestimation of third party candidates). Both a poll a month before the election and one conducted a week before showed nearly same breakdown of partisan voters (42/43% Dem, 40/42% Ind, and 18% Rep). The fact that the polling was stable and the breakdown of voters by partisanship are stable make me think the partisan breakdowns are VERY close to being correct.
So the question to me was how did turnout in 07 by partisanship compare with the actual registered voter breakdown. We would naturally expect an increase in the Dems/Reps percentages due to Massachusetts Open Primary system (people who have allegiance have reason to say they are indies). Breaking down by party ratios (comparing SurveyUSA's numbers and those from the Secretary of State taken 3 months after the election... the closest in time total) Dems got 130% of their registered total, Reps got 139% of their registered, Indies (unaffiliated) got 73% of their total. How does that compare with the Rasmussen poll? Per the Rasmussen poll, Dems get 140% of their registered total, Reps get 180% registered percentage, and Indies got 53% of their total.
In other words, you see a real depression of the indie percentage as compared to 07. IMHO, this makes little to no sense. Sure, the idea that Reps are turning out in an even higher percentage than registered fits the narrative that the base is "fired up". But why the devil are the supposedly "down and out" Democrats also showing up at a higher percentage compared to the 07 special election, when they were flying high (even if Tsongas wasn't exactly a barn burner)? The higher percentage of Reps should come at the expense from Dems, not indies (or at least equally from both).

SurveyUSA asks "do you consider yourself a Dem, Rep, or Indie". That should lead to less indies than the Rasmussen poll... in fact, it leads to more.

I'm not buying.


---------------------------
Put more clearly:

Here's what I did. I did compare the two SUSA polls from the 5th district (http://www.surveyusa.com/client/PollPrint.aspx?g=04fe9b30-61de-4f66-9c29-2de5004b960f&d=0 and http://www.surveyusa.com/client/PollReport.aspx?g=9cd85212-ab1e-46ff-91b0-ef66436384e1) to the Rasmussen poll as claimed partisan id... sorta.

I searched for the closest party ID from Secretary of State Galvin (http://www.sec.state.ma.us/ele/) by time proximity to the 2007 special election (October of 07) figures. I found the figures put out prior to the Democratic Presidential Primary in 08 in January 2008 or December 07... you see two different dates on this document (http://www.sec.state.ma.us/ele/elepdf/pp_county_town_enroll_breakdown_08.pdf).

I went and copied the enrollment for each of the 29 municipalities in the district http://en.wikipedia.org/wiki/Massachusetts's_5th_congressional_district . For Wayland, which has only 3 of 4 precincts in the district, I found how much these 3 of 4 precincts registered voters made up of the municipality's registered voters (see page 11 http://www.wayland.ma.us/selectmen/Town%20Report%202008.pdf). Granted those Wayland precinct numbers are from June 08 (six months after my numbers, but I'm guessing they are quite close... in either case the municipality is pretty small).

I compared the Surveyusa polls' partisan registration (different I know) to the registered voters' statistics from the Secretary of State. And came up with the ratios for 2007 (only SurveyUSA in 2007), I listed below.

Then I took the Ramussen poll in 2010 and compared it to the statewide statistics put out just prior the general election last year, late 2008 (the latest available figures).

You'd expect relatively similar ratios (or rasmussen's to more closely match the secretary of state's due to how party id is asked) because the district and state are relatively the same in terms of party id (or partisanship) and demographics (the 5th is slightly more hispanic and slightly less black).

Both are special elections with relatively little press coverage and low voter enthusiasm.

And both had competitive primaries on the dem side... and not on the Rep side (in fact Brown's and Ogonowski's primary victories were both by 80%).

But you don't get what you would expect.

____________________

poughies:

That said...

If you use statistics more similar to what my normally would be expected... and you apply it to Rasmussen's crosstabs....

Brown is actually in the lead (crosstabs not available to non-paying public)...

Then again, I don't think Brown does so well among Democrats and Indies, and relatively poorly among Republicans.

____________________

Farleftandproud:

I know that Rasmussen is full of shit when it comes to party ID. I seriously cannot understand what format they are using. The average is about 7-8 percentage higher among Democrats compared to Republicans. I think Rasmussen bends the rules; they poll a lot of Independents who are likely voters and when you can pick and choose what you want to use than I suppose there are almost as many Republicans as Democrats.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR