Articles and Analysis


What to believe about Massachusetts Senate Polls

There has been a wider than normal range of polling results in the last two weeks from the Massachusetts Senate special election. This has been further clouded by a number of leaked internal polls and polling by relatively unknown and unproven pollsters, some partisan but others not.  And most importantly, the rapid shifts in the race, reflected across all the polls, makes this a fast moving target. So let's take a moment to consider what we could reasonably conclude based on the data.

But no matter how you slice the data, the only reasonable conclusion is that Scott Brown has moved from well behind to a lead somewhere between 4 and 11 points.

The chart above shows all the polls we have available as of 12:36 a.m. Monday morning. That includes new PPP and Pajamas Media/CrossTarget polls released late Sunday evening. The chart also includes the leaked polls, mostly from the Coakley campaign but one from Brown as well.  These leaked polls are NOT included in most of the estimates above, though they are not out of line with the rest of the data. 

So what might you believe about these data? You could refuse to cherry pick the polls. That has long been our view here at Pollster.com. Our job is to summarize the trends as best we can, without partisan favor. If you do that, we get a 8.8 point Brown lead.

Perhaps you only trust non-partisan polls. Then the Brown lead is 6.8 points.

Maybe you are a Dem, who doesn't trust the Republican pollsters. Then Brown leads by 6.5 points.

Or you are a Dem who doesn't trust the non-partisan pollsters either and who does believe in the leaks from the Coakley campaign. Then Brown's lead is 3.8 points. (This is the only estimate that includes the leaks.)

Or you are a Rep who trusts GOP and nonpartisan polls only. Then Brown leads by 11.3. (There aren't enough Rep polls to run a Rep only estimate to parallel the Dem only, but I'd think an 11 point lead would be satisfying enough for Reps.)

There may be other ways to cut these data (IVR vs conventional phone, pollsters you've heard of vs ones you haven't) but it seems quite unlikely that any but the most selective reading of these data can find that the race remains a dead heat. Brown has a lead, as of Sunday night.

Let's back up a step to look at the data without the clutter. Here are just the polls, no trends fit.
Without the lines it is quite clear that the movement has been sharply towards Brown. Trace out what you like, ignore what you don't like, in the early polls Coakley is convincingly ahead. Then between about day -8 and -5 the polls are balanced above and below dead even. Since then no poll has shown a Coakley. 

In my models in the first chart, I use linear fits rather than our usual local regressions. The reason is there are still not very many polls,and once we subset them by party there simply aren't enough cases to get good local regression fits. That subsetting is the main point here. But it also turns out that the local regression on all the data isn't very far from the linear fits I use above. Here is the comparison:

The blue trend is our standard estimate, and it wiggles a bit due to only 12 cases. If we use a bit less sensitive local regression, we get the black line. And the red linear fit isn't very far from either of the two local fits. So I'm willing to give up some flexibility in the fit for a bit more robustness, and especially the ability to fit the models by party of pollster that was the lede above. 

Finally but significantly, we are seeing more pollster variation in this race than normal. If we look at the residuals around the trend estimates, past experience with 2004, 2006 and 2008 state and national contests has pretty consistently found that most of the polls (about 95%) fall within +/- 5 points of the trend estimate. Now that is an empirical observation, not a theoretical one. But it has been generally consistent in our data. How do these polls compare?
Only half of the current polls are inside +/-5 points of the linear trends. The number of polls is small, and this race is more dynamic than most. But one has to wonder about the problems of polling in a special election, the role of partisan and new players in the polling and the heavy use of IVR polls. This is much more variation in polls than we normally see in general elections.

Let's also recall the NY-23 special election, which was not polling's finest hour. The last three polls there had Hoffman up by 5, 5 and 17 points. Our final trend estimate based on all the polls had Hoffman up by 5,  41.8 to 36.8. 

Polling special elections is hard. Tuesday we'll see how hard, and who was good and/or lucky.



Your brief mention of the Hoffman 23rd polling experience ignores just how much can be learned from that experience as we approach tomorrow's vote. In the Hoffman/Owens polls it was a special election, the closing days saw a strong trend to Hoffman, there was an angry & energized conservative base, the Dems parachuted in last minute heavy weights to campaign for Owens, and PPP had a last minute IVR poll that gave Hoffman a seemingly insurmountable 5 point lead that capped a dramatic charge from hopeless underdog to seemingly sure winner. And the media went into a frenzy over the juicy narrative about the dire implications Hoffman's stunning & historic upset would have for Obama, healthcare reform, the midterms, and the Dem's certain demise as a party and force in American politics. As Yogi would say, it seems like Deja vu all over again.


tom brady:

Of course, the other element that makes comparison with NY's 23rd problematic is the last minute decision in that race by scozzafava to drop out and endorse the Democrat Owens. Nothing comparable is going to happen here. Democrats ought not put much faith in that scenario. Looks like a different set of electoral dynamics here.



Perhaps, but there also many analogous factors that could contribute to the polls being so volatile. Another post on this site references the non-response bias, which might normally be representative across the sample, but may be skewed to one side in these types of elections where there is a 'motivated by anger' population eager for the chance to sound off versus a 'tired of all the noise' population who don't want to answer questions but just want to vote and get it over with. And the Suffolk Poll that first showed Brown taking a lead, with a very odd age distribution, supports that hyppothesis since the age bubble was the 45-59 range. That is the age demographic of disaffected, angry tea party types who are itching to be heard as we've seen by their town hall shenanigans. I think the Scozzafava (lord what a name) influence in the NY 23rd was that her exit left a very large block of undecided heading into the election compared to the Mass. situation, and they obviously broke more for Owens. So yes, the trends in Mass have less of an unknown about them that could potentially result in a surprising outcome relative to the polling.



Their are many problems with trying to do a direct comparison of this to NY 23rd special election:

1. That was 2 months ago. Obama's approval was still positive.

2. Healthcare was still iffy at best. The secret negotiations, sweetheart deals, and bribes hadn't come out yet. Even the most cynical didn't expect the Democrats to stoop as low as they did to shove it through.

3. Like tom brady said above - scozzafa's last minute betrayal made a diference.

4. Brown is a good campaigner - Hoffman stunk.

5. Coakley is one of the worst campaigners ever and her arrogance in just sitting back waiting for a coronation fanned the flames of discontent.


tom brady:

Is Rasmussen polling the Senate race one more time? If so, do they expect to release results today?


Gary Wagner: All of what you say is true. However, it is also true that NY23 was an R+1 registration district, but MA is a D+20 or so state.

Also, the PajamasMedia poll of 1/14 showed 71% definitely voting. Their poll of 1/17 showed 96%! Turnout will be huge tomorrow; possibly close to the 72% from 2008.



Rasmussen's site has been down all morning. Not characteristic at all.

I smell sabotage. And I am even much of a conspiracy theorist:-)

If course, there is nothing preventing Scott from sending results to sites like this one.


Actually, twitter has been rife with rumors (since last night) that Kennedy may drop and endorse Brown. FWIW.

I also think that the OBama rally may have hurt Coakley - she looked very awkward speaking and constantly consulting her notes (which probably said "now, roll up your sleeves" etc).

The sudden appearance of Coakley campaign posters (none were actually on lawns) looked much too planned to have been spontaneous - every single person in the rows behind Coakley on camera held an identical Coakley sign and carefully leveled it facing the camera, a vivid contrast to the wild spontenaiety of Brown supporters with their home-made placards. It was a machine-made rally, and one wonders how many would have attended were it not to see Obama.

I personally visited Brown HQs in Worcester yesterday to see for myself what's up.
I found something very different from the tea-party neocons Coakley has tried to portray, and I think most people sense the truth there.

Here's a stream of videos I captured from Brown HQ, see for yourself http://qik.com/gop



A key factor here is voter turnout and the definition of 'likely voter'. The attention this race is getting in the national press and its perceived closeness is going to motivate many voters who would have otherwise stayed home on Tuesday.

The irony is that Brown's strength in last weeks polling may be the very thing that keeps him from being elected. I predict Coakley's going to win by 5-10 points.



Dan: The difference between the current race and NY 23rd is that when "Republican" Scozzofavi realized she was losing she turned-coat and endorsed the Democrat candidate at the last possible minute.




Yes - you make a point. You also have to remember that Hoffman was not the Republican on the ticket. Owens won that race with 49% of the vote. The non-democrat vote was 51%. You would have expected the Democrat to get 49% in a R+1 district. He did.

I don't have any illusions that MA has turned into a Republican state. Brown is riding one of the strongest anti-establishment waves since the Reagan Reveloution. 20% of the people who voted for Obama plan to vote for Brown. They are disillusioned with a president who promised to change the way Washington politics takes place and he has made it worse than they have ever seen in their lifetimes.

todji: There's something to be said in Brown's strength bringing out the anti-Brown vote that might have stayed home otherwise. I don't agreee with your prediction though. I predict Brown by 2 points.



Gary: I can't pretend my prediction is anything other than a guess. We'll have to wait until Wed. to see which of us is closer.



Scozzafava was polling a lot higher than Kennedy is at less than 2%. So a Kennedy endorsement isn't going to have nearly the same effect as did Scozzafava's.



todji: Same here. I think we might be witnessing something here that defies predictions and polls. That's probably why we are seeing Rasmussen showing the highest support for Coakley of any of the polls. Very curious.



Gary: That was pretty much Nate Silver's conclusion as well.


pete fabrizio:

The Rasmussen poll is also the oldest at this point. Since they released the Brown -2 on January 11th, Brown has gained 4 in PPP, 8 in R2K, 4 in ARG, and three one-off polls have come out showing him with a 9+ point lead (I'd count the PJM polls in that category as one). The only polling that sounds favorable to the dems are the alleged leaked Coakley internals, which supposedly give her a two point lead -- the same as a week ago.

At this point, I think there is no way the polls can be construed as supporting a Coakley victory (although Nate S. certainly seems to try). Which is not to say she can't win: there could be systematic errors with the polling methodology and/or voter models in this race, and of course there's some chance that Coakley makes a last-second comeback. But the war for the best pre-election data is over.



Rasmussen decided not to run another poll, instead he said if voter turn-out is high it will likely be Coakley and if not it will likely be Brown. I think this goes to show just how hard this race is to call.



They're getting suicidal at Democratic Underground (link to thread).Quote from thread:

15. You know, I really wish you wouldn't post this so the repubs and lurkers can see it.
I for one don't want to see supposedly negative for our candidate.

DU makes DailyKos look moderate. Even they're expecting a loss.

Boo hoo.


I think that it's just going to boil down to turnout. Given the large majority of Democratic voters in Mass. I think a large turnout favors Coakely. Not that this is an epiphany, most of the pundits have stated the same. Given the national attention given to this race it is likely that those who have an issue with the Obama agenda are likely to show up and vote believing this is a chance to stop legislation such as health care. That is a powerful motivator and may be the difference.


Terence Haake:

Hoffman is not Scott Brown, Hoffman was a geek with almost no political skills allmost beat a policticaly connected inside the DNC Beltway liberal. Scott Browns Personality and polish destroyed Coakely arogance and sence of intitelment and that Dead Tedy would want her in the Senate as the 60th vote. Scott Brown will come out the winner between 2-10% over Coakley with Obama and Clintoons coatails basicaly stabbing her in the back because they are not relevent. I saw a half Empty room when Obama showed up for her his coatails have turned on Demomicrats. They bused in the people to fill the rallys empty hall. Very few locals attended her Obama/Coakely event!! Coakely is Hoffman her skills aren't much better. Hoffman probably had the right message but horrible looks and poor communications he almost beat DNC political pro. Brown is a winnner, Coaklely is a whinning sniveling little minded liberal Democrat!


Post a comment

Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.