Guest Pollster | January 17, 2010
Topics: 2010 , Likely Voters , Martha Coakley , Massachusetts , non-response bias , representativeness , Scott Brown
Mike Mokrzycki is an independent consultant who was the founding director of the Associated Press polling unit. He may be reached at firstname.lastname@example.org. His guest contribution is cross-posted from his blog, MJM Survey Musings.
One thing is certain about the polling in the last days before Tuesday's special election in Massachusetts to fill the late Ted Kennedy's U.S. Senate seat: Someone's going to end up being very, very wrong.
Polls completed in the past week and recorded at Pollster.com range from a 14-point lead for Democrat Martha Coakley - just weeks ago considered a shoo-in in heavily Democratic Massachusetts - to a 15-point advantage for Republican Scott Brown, who has become a darling and major fund-raising beneficiary of conservatives nationwide.
I'm not going to do a deep methodological dive into all these polls to try to explain the differences. Pollster.com and Fivethirtyeight.com have done their usual stellar job with that already, including analyzing the extraordinary uncertainty inherent in trying to determine who really will vote in this mid-January special election.
I will try to provide a little perspective as someone on the ground in Massachusetts who also knows a thing or two about polls.
My hypothesis: While Brown supporters clearly are more enthusiastic than Coakley backers, that may serve him relatively better in the pre-election telephone polls than it will Tuesday.
I've lived in Massachusetts on and off since 1980 and I can't ever recall Republicans here as energized as they are now. Sure, they had a 16-year run in the governor's office despite the state's overall leftward tilt. But Bill Weld, elected in 1990, was fairly unusual - socially liberal enough that "Weld Republican" became its own label. Paul Cellucci sure didn't inspire a lot of passion and I can't say Mitt Romney did either, with his eye on the White House all along. Brown seems an agile campaigner but I don't think his personal charisma is what's charging up Republicans here and elsewhere; rather, it's the once almost-unthinkable notion that any Republican might actually win the seat Ted Kennedy held for nearly half a century, especially with such extremely high stakes for policy and politics nationally.
This enthusiasm is abundantly evident in internal data from numerous polls. I'd add a couple anecdotes: I don't put stock in lawn signs but when you see a voter (like someone on the main street in my town) posting a handmade Scott Brown placard, or an ice cream stand using its roadside sign to advertise "VOTE FOR SCOTT BROWN," it may be an indication something beyond rote partisanship is at work.
This race has been Coakley's to lose, and she's seemingly been doing her best to do that. The most recent example was in a radio interview the other day when she called Curt Schilling - famous for helping pitch the Boston Red Sox to a long-awaited World Series championship in 2004 on an ankle stitched together and visibly bleeding through his sock - a New York Yankees fan, of all things. In little more than the time it used to take a Schilling fastball to reach the plate, his recorded voice was on my phone telling me this faux pas was proof Coakley was out of touch with Massachusetts voters. Silly, weighed against the import of issues such as health care reform? Perhaps. But - last baseball metaphor, I promise - Coakley served up a big fat meatball and I sure don't blame the Brown campaign for hitting it out of the park.
Schilling's was one of countless phone calls we've gotten on this race since before the primaries last month. Many have been "robo-calls" like his (as I write this paragraph I just got one from Brown's daughter), though plenty feature live human beings (like someone from Coakley's phone bank who called as I started writing this post).
At this point it's hard to blame people in Massachusetts for screening incoming calls even more so than usual. For years there's been plenty of screening, part of the reason why response rates for all kinds of telephone polls have declined dramatically. (An article in the Winter 2009 issue of the journal Public Opinion Quarterly (subcription required) gives response rates for numerous respected telephone polls it cites and many of them barely crack 10 percent. A response rate greater than 20 percent now is extraordinarily good.) Response rates are even lower for automated polls, which use a recorded voice for interviews and require respondents to punch in answers on the touchtone keypad.
But - and this is an important "but" - a growing body of research indicates decreasing response rates have not hurt the accuracy of survey estimates. That happens when there's no systematic difference between those who cooperate and take the survey and those who decline.
I'm thinking the Massachusetts Senate race may be a case where we do see non-response bias in surveys. It comes down to relative enthusiasm for the candidates. It's tough to prove, but I'd venture a guess the dynamic works like this:
- Republicans are excited Brown might win and thus more likely to
answer their phone and listen to political messages -possibly be
invited to take a survey - when the phone is practically ringing off
the hook with such calls. I suspect they'd be particularly enthused to
participate in a poll and tell the world they're voting for Brown, to
help build the sense he has unstoppable momentum. These folks certainly
will vote but there's no upside in Brown's election-day numbers
compared to the pre-election poll estimates.
- Democrats may be demoralized and scared after several weeks of Coakley campaign missteps and bad headlines. They may not be all that eager to pick up the phone for political calls. They also might be more skeptical of or angry about polls since they've been such downers for Coakley and President Obama lately, and thus, I would speculate, more likely to take a pass if invited to participate in one. None of that means these folks are less likely to vote, though - by now any sentient Democratic-leaning voter will know Coakley needs all their votes, and what's at stake. They might not be happy about how Coakley has run her campaign but they'll still be motivated to vote by a desire to deny a Republican the chance to do serious harm to Obama's agenda from Ted Kennedy's old seat. Obama is in Massachusetts today to remind them of exactly that (not that they're necessarily all that enthused about him at this point, either).
Of course, truly independent or "swing" voters are another vital factor, and if Brown wins enough of them he could overcome the inherent Democratic advantage in Massachusetts. But I'd think enthusiasm, or lack of it, would be more of an issue among stronger partisans.
In pollster speak, what this boils down to is "differential non-response," where one candidate's supporters are more likely than the other's to take a survey. It's suspected to be a big reason why exit polls in recent years have tended to overstate support for Democratic candidates. In the Massachusetts special Senate election I suspect it's inflating the Republican's poll numbers. Coakley has room to outperform the polls Tuesday even if her natural base is motivated by nothing more than fear of what would happen if her opponent pulls off an historic upset.