The events of the week delayed my pointing to a terrific resource made available to pollsters and polling junkies over the weekend by Public Opinion Quarterly, the academic journal published by the American Association for Public Opinion Research (AAPOR). Their special issue on the 2008 Presidential Election is now posted online and access is totally free.
It leads with the article previously teased on Pollster.com by co-author Mike Mokrzycki on exit poll measurements of cell-phone only voters and implications for future polling. It also includes many more articles of interest to Pollster readers, including a review of poll performance in 2008 by Mike Traugott and Chris Wlezien, a look at how predication markets compare to polls in forecasting outcomes by David Rothschild and much, much more.
Washington Post / Kaiser Family Foundation / Harvard School of Public Health
1/20-21/10; 880 Special Election voters, 4% margin of error
242 non-voters, 8% margin of error
Mode: Live telephone interviews
State of the Country
31% Right Direction, 63% Wrong Track
Brown voters: 15 / 81
Coakley voters: 49 / 44
Brown voters: Was your Senate vote more for Brown or more against Coakley?
70% For Brown, 25% Against Coakley
Coakley voters: Was your Senate vote more for Coakley or more against Brown?
57% For Coakley, 40% Against Brown
Was one reason for your vote for Senator to express support for Obama, to express opposition to Obama, or was Obama not a factor in your choice?
28% Support, 23% Oppose, 48% Obama not a factor
Was one reason for your vote for Senator to express support for the Democratic agenda in Washington, to express opposition to this, or was the Democratic agenda not a factor in your choice?
34% Support, 35% Oppose, 29% Not a factor
When senator-elect Brown goes to Washington, do you think he should mainly work with the Democrats to try to get some Republican ideas into legislation or should mainly work to stop the Democratic agenda?
82% Work with Democrats, 11% Stop Democratic agenda
On health care: 70% Work with Democrats, 28% Stop changes
Overall, given what you know about them, would you say you support or oppose the proposed changes to the health care system being developed by Congress and the Obama administration?
43% Support, 48% Oppose
Generally speaking, do you strongly favor, somewhat favor, somewhat oppose or strongly oppose the health care reform plan proposed by President Obama and the congressional Democrats?
40% Somewhat/Strongly Favor, 58% Somewhat/Strongly Oppose (chart)
The January Kaiser Health Tracking Poll, conducted before the Massachusetts Senate vote, finds opinion is divided when it comes to the hotly debated legislation, with 42 percent supporting the proposals in the Congress, 41 percent opposing them and 16 percent withholding judgment. However, a different and more positive picture emerged when we examined the public's awareness of, and reactions to, major provisions included in the bills. Majorities reported feeling more favorable toward the proposed legislation after learning about many of the key elements, with the notable exceptions of the individual mandate and the overall price tag.
For example, after hearing that tax credits would be available to small businesses that want to offer coverage to their employees, 73 percent said it made them more supportive of the legislation. Sixty-seven percent said they were more supportive when they heard that the legislation included health insurance exchanges, and 63 percent felt that way after being told that people could no longer be denied coverage because of pre-existing conditions. Sixty percent were more supportive after hearing that the legislation would help close the Medicare "doughnut hole" so that seniors would no longer face a period of having to pay the full cost of their medicines. Of the 27 elements of the legislation tested in the poll, 17 moved a majority to feel more positively about the bills and two moved a majority to be more negative.
Which comes closer to describing your own views? Given the serious economic problems facing the country we cannot afford to take on health care reform right now OR it is more important
than ever to take on health care reform now?
54% Take on now
39% Cannot afford
As of right now, do you generally support or generally oppose the health care proposals being discussed in Congress?
42% Support, 41% Oppose (chart)
Do you think _____ would be better off or worse off if the president and Congress passed health care reform, or don't you think it would make much difference?
You and your family: 32% Better off, 33% Worse off, 29% No difference
The country as a whole: 42% Better off, 37% Worse off, 12% No difference
As far as you know, is the health reform bill currently being discussed in Congress expected to increase the federal budget deficit over the next ten years, decrease the deficit over the next
ten years, or is it not expected to have much impact on the deficit?
60% Increase, 15% Decrease, 17% No impact
continue sort through the rubble of data from the Massachusetts special
election.As I posted
yesterday, some consensus has emerged that Coakley underperformed consistently,
within her own base as well as with independents.And the consensus on cable news suggested opposition
to both Obama and health care reform drove the election.But the data this week was far from
consistent on this latter point.Before
we use the recent election to inform the health care debate and 2010 campaign strategy,
it's worth examining the data more closely.
Some polls show "sending a
message" or "stopping" Obama not the most powerful driver
polling firm Hart Research conducted an election day
survey that showed stopping Obama to not be front-and-center.They found fewer than half (42%) of voters
felt "sending a message" about Obama going too far was either the single most
or a very important quality they looked for in a Senator, far behind
strengthening the economy and "controlling health care costs and covering the
uninsured."Further, as the memo points
out, "Even Brown voters were more are more concerned about a lack of change
(50%) than about trying to make too many changes too quickly (43%)."Further, by a margin of 2-to-1, voters said
they were voting "for the best candidate" instead of "to send a message to
Many polls show voters turning
out to support health care
night poll by Coakley's pollster Celinda Lake asked specifically about
whether one's vote was to show support for health care or to show
opposition.A plurality (46%) said it
was to show support.The Hart survey
also said those who knew Brown's position on health care were just as likely to
vote against him because of it (39%) as vote for him (41%).
Hart poll, this
Rasmussen poll, and the Lake poll, all showed that voters who named health
care as their top concern were more likely to support Coakley.And all three of these polls showed voters for
whom the economy was most salient gave Brown the advantage.
One Republican poll, however,
disagrees on both points
pollster Fabrizio also conducted an election night survey,
and as noted here,
differed from his colleagues as to what drove the vote.His poll shows Brown voters responding in an
open-end that health care was the single biggest factor in their vote.For Coakley voters, health care came in after
a more vague "I'm a Democrat." He also
finds a plurality (46%) of Brown voters saying their vote was to "send a
message to Washington," with about as many (43%) claiming it was "for
Brown."But interestingly, more
Democratic and independent Brown voters claim their vote was to send a message
(50% and 52%, respectively), while a majority of Republicans (56%) were voting
A progressive-sponsored poll looked
at different questions altogether
consortium of progressive groups (PCCC, MoveOn, and Democracy for America) also
commissioned an election
night survey, conducted by Research 2000.Their survey had a unique methodology; they surveyed Obama voters who
stayed home, and Obama voters who voted for Brown.They found a plurality of Obama/Brown voters
feel Democrats are not "fighting hard enough to challenge the Republican
policies of the Bush years."And like
many of the surveys above, they found these Obama/Brown voters, across all
parties, to say the economy was more important to their vote than health care.
while the poll shows a plurality of Obama/Brown voters oppose health care
reform (48% oppose), more of those who oppose think it doesn't go far enough
(18% of all Obama/Brown voters) than think it goes too far (11% of all
Obama/Brown voters).However, even more
health care opponents aren't sure whether it goes too far or not far enough
(20% of all Obama/Brown voters).
So what are the lessons from
some takeaways from the post-election polling this week.
Exit polls make it
easier to form a consensus about what happened.For all the perennial complaints about the shortcomings of exit polls,
the fact is they do provide us with an unbiased source of data to help distill
meaning from election results. In their absence, we are left with
competing claims from surveys conducted by mostly partisan pollsters.In this case, the claim that opposition to
health care defeated Coakley, while widely adopted, is not a consistent finding
in public post-election polling.
care was important to Coakley voters.Whether it
was the most or second-most important issue to Coakley voters, it's clear
across surveys that it was indeed important.So attitudes toward health care motivated both Coakley and Brown voters.It's a data point that might have helped
Howard Dean in this debate
with Chris Matthews this week.
motivated Brown voters.Brown voters were motivated by a
myriad of factors.While pollsters
disagree as to how much health care or "sending a message" drove Brown voters, there
is consensus that he was simply more popular than Coakley.The Hart and Rasmussen surveys both found
Brown to be substantially more popular than Coakley.Brown also consistently had the advantage
over Coakley among voters concerned about the economy.
health care begets opposition.The Research 2000 survey shows those opposed
to health care are not quite sure why they are opposed.
Question wording on
health care continues to evolve.The Fabrizio survey used open-ended questions
to determine what the most important factors to the vote.This may produce different responses than the
closed-ended questions in the Hart, Rasmussen, Lake, and Research 2000 polls.And maybe Washington shorthands like "going
too far" and "not going far enough" have different meanings to voters still
sorting through health care reform's specifics.
newly released Washington Post/Kaiser/Harvard poll, conducted after the
election is consistent with some of the findings discussed above.First, it shows opposition to Obama was not the
biggest motivator for Brown voters.Over
half (52%) said "Obama was not a factor" compared to 43% who said
their vote was "to express opposition to Obama."Brown voters are also evenly divided between
whether Brown should work with Democrats on health care reform (48%) or stop
changes to health care from happening (50%).Also worth noting, a full 37% of Brown voters said they are dissatisfied
or angry about the "policies offered by the Republicans in
Congress."hardly a national mandate
for Republican takeover.
poll also shows health care to be salient to both Coakley and Brown
voters.In fact, it appears that health
care is the singularly most dominant issue with Coakley voters (compared to the
economy, "the way Washington is working," candidate personal qualities,
government handling of banks, and others), while Brown voters are a bit more
divided amongst their top-tier of issues.
State of the Country
34% Right Direction, 52% Wrong Track (chart)
Overall, given what you know about them, do you support or oppose the proposed changes to the health care system being developed by Congress and the Obama Administration?
44% Support, 56% Oppose (chart)
1/20/10; 1,010 adults, 4% margin of error
Mode: Live telephone interviews
Which comes closest to your view of President Obama during his first year in office - you are pleased with the progress he has made on the problems facing the country, you are disappointed with the progress he has made because you thought he would have made more progress, or you are upset because you think he is taking the country in the wrong direction with his policies?
39% Pleased, 20% Disappointed, 37% Moving country in wrong direction
Which comes closest to your view- the president and Democratic leaders in Congress are right to make healthcare their top priority at this time, healthcare is important but there are other problems that Congress and the president should address first, or healthcare should not be a major priority for the president and Congress?
32% Right to make health care top
46% Important but should address other problems first
19% Should not be a major priority
Do you think the country would be better off if the Republicans controlled Congress, or if the Democrats controlled Congress?
36% Republicans, 40% Democrats
As you may know, Republican Scott Brown won the special election to the U.S. senate seat from Massachusetts formerly held by Ted Kennedy. Which comes closest to your view of what this result means - it reflects political conditions in Massachusetts and doesn't have a larger meaning for national politics, or it reflects frustrations shared by many Americans and the president and members of Congress should pay attention to it?/em>
18% Political conditions in MA
72% Reflects frustrations by many Americans
What do you think President Obama and the Democrats in Congress should do now concerning a healthcare reform bill -- should they-- continue to try and pass the healthcare bill now being worked out by Democrats in the House and Senate suspend work on the current healthcare bill the House and Senate are working on and consider alternative bills that can receive more Republican support?
39% Continue with healthcare bill
55% Suspend work and consider alternatives
Fox News/Opinion Dynamics
1/12-13/10; 900 registered voters, 3% margin of error
Mode: Live telephone interviews
Obama 47%, Romney 35%
Obama 55%, Palin 31%
Obama 53%, Gingrich 29%
Obama 48%, Tea Party candidate 23%
Who do you want to win this year's congressional elections -- the
Democrats or the Republicans?
38% Democrats, 37% Republicans
Do you think it would be good or bad for the country if all the current members of Congress were voted out of office -- including your own
representative -- and all new people were elected this November to get a fresh start?
43% Good, 44% Bad
If the 2012 presidential election were held today, would you...
23% Definitely vote to re-elect Barack Obama
20% Probably vote to re-elect Obama
11% Probably vote for someone else
36% Definitely vote for someone else?
On a morning full of big political news -- the apparent death of the health reform bill, a Supreme Court decision ending restrictions on corporate campaign spending and John Edwards admission that he fathered an out of wedlock child -- I spent the morning attending a briefing something even more exciting: What's in store for execution of the 2010 Census.
OK, maybe not that exciting, but the session sponsored by the DC chapter of the American Association for Public Opinion Research (AAPOR) and the Pew Research Center did provide a fascinating glimpse at one of the government's decennial effort to count and gather demographic data on its citizenry.
While newly installed Census Director Robert Groves did not break any news with his brief presentation, he conveyed a sense of the "massive undertaking" involved in executing the Census and passed along some information that would interest pollsters and data geeks:
In March the Census will mail a census form to every residence in the United States, a procedure used by the Census since 1960.
To improve the rate of response to the initial mailing, the Census will run a paid advertising campaign as it did 10 years ago. Groves added this statistic: "For every one percentage point that we raise the mail response rate through this advertising campaign, we will reduce the total cost to the Census by about $85 million," costs that they incur by sending follow-up mailings and in-person enumerators to gain full response (a video of highlights from the launch of the ad campaign is posted here).
The Census is currently conducting a daily telephone tracking survey to monitor what Groves described as "key predictors" from prior research of what predicts likelihood of participating in the 2010 Census. They then run "predictive models" and "watch how the predictors are changing" overall and within key subgroups and, if necessary, tweak their advertising buy or messages appropriately Groves showed a tracking chart indicating that awareness started to climb following the New York City roll-out of the Census "Road Tour" (seen in the video below).
As also indicated in the survey released by the Pew Research Center yesterday, younger Americans -- those between 18 and 29 years of age -- are the "laggard group" in terms of reported awareness and intent to participate. Groves, the noted expert on response rates in surveys, voiced a caution we rarely hear about telephone surveys: These data on younger Americans "are subject to great misinterpretation" since they are from the respondents of a random digit dial (RDD) telephone survey. "The proportion of all people sampled that became respondents is much, much, much, much, much, much lower than we'll ever get in the census, so these [younger respondents] are the cream of the crop" in terms of their willingness to participate in a survey, and thus perhaps, in the census itself. "The usefulness of this [survey]," he added, "is to watch this over time, to see if things are moving."
Starting at the end of March, the Census will launch a tool on its web site that will allow anyone to monitor real time updates of the participation rate, featuring a thematic "heat map" that will display regional variation. You will also be able to "drill down" to see similar mapping for individual counties or zip codes, or create on screen comparisons between localities so "New York could compete with L.A." (or perhaps Washington DC with Dallas?).
The Census is also undertaking a Census in the Schools initiative with the assistance of Sesame Street characters Rosita and Count von Count, who Groves describes wryly as his "senior technical adviser."
Some consensus has emerged from Tuesday's Special Election. First, Coakley underperformed consistently. Absent consortium exit polls, but with no shortage of post-game analyses, we see Coakley underperformed not just with independents, but with the Democratic base. This debunks a common explanation for her loss--that Massachusetts simply isn't ready for a woman Senator (or Governor).
First, Coakley had the same gender gap as Obama did in Massachusetts in 2008. According to 2008 exit polls, there was a 20-point difference between Obama's advantage with women (+32) and his advantage with men (+12). Similarly, according to a post-election survey and analysis by Coakley's pollster, Coakley led by four points with women, but trailed by 16 points with men. So Coakley ran behind Obama by the same margin with both women and men.
Second, Coakley underperformed dramatically among liberal-leaning groups of women. The Lake poll shows her having a 17-point advantage with unmarried women. Nationally, Obama had a 41-point advantage with this group. According to the Lake analysis, Corzine had a larger advantage with this group, while the really underperforming Deeds had a smaller advantage. Another liberal-leaning group, college-educated women, were divided evenly between Coakley and Brown (50% for each). While there aren't public numbers for Obama among Massachusetts voters, nationally in 2008 he led by 27 points among white college-educated voters of both genders.
These figures suggest that Massachusetts sexism did not hold Coakley back, unless that sexism is as prevalent in liberal women as it is with other groups. That seems unlikely. Coakely underperformed consistently, throughout the state, and across the demographic spectrum. Campaign tactics, the mercurial nature of a special election, a volatile national climate, and the mobilization of the Republican base were sufficient to yield a Republican upset.
Democratic elites in Washington are generally optimistic about the trajectory of the US economy, with 64% saying that the economy has bottomed out and is getting better. Only 19% of Democrats feel that we're stuck at the bottom, and only 15% think the worst is yet to come.
Republican elites in Washington are much less optimistic. One third (33%) think that the economy hasn't bottomed out yet and will get worse. Another 25% think the economy has bottomed out and is not getting any better. 41% think the economy has bottomed out and is on the upswing.
Via John Sides, David W. Brady, Daniel P. Kessler, and Douglas Rivers have published an op-ed in the Wall Street Journal that is likely to fuel Democratic panic in Washington over health care:
The majority party normally loses seats in midterm elections, but the Republican resurgence of recent months is more than a conventional midterm rebound. How can a little known Republican run a competitive Senate campaign in Massachusetts? The culprit is the unpopularity of health reform, and it means that Democrats will face even worse problems later this year in less liberal places than Massachusetts.
We have polled voters in 11 states likely to have competitive Senate races in November on how they feel about health reform and how they might vote in November...
Health reform is more popular in some of these states than in others. Where it's popular, Democratic candidates don't have too much of a problem, but where it's unpopular--and that includes most states--the Democratic Senate candidates are fighting an uphill battle...
Support for the Republican Senate candidates in these races is closely related to voter opposition to the health-care Senate bill...
How do we know that it's the health-reform bill that's to blame for the low poll numbers for Democratic Senate candidates and not just that these are more conservative states?
First, we asked voters how their incumbent senator voted on the health-care bill that passed on Christmas Eve. About two-thirds answered correctly. Even now, long before Senate campaigns have intensified, voters know where the candidates stand on health care. And second, we asked voters about their preference for Democrat versus Republican candidates in a generic House race. As in the Senate, the higher the level of opposition to health reform, the greater the likelihood that the state's voters supported Republicans.
Brady and Rivers are highly respected political scientists (I'm not familiar with Kessler), but I'm not sure we can draw strong conclusions from these data. Since health care passed on a perfect party line vote in the Senate, it's relatively easy to know where an incumbent stands on the issue. And given the salience of the health care debate, the correlation between state opposition to health care reform and support for Republican senate candidates is (a) not surprising and (b) not necessarily causal (especially given that those are aggregate measures).
I tend to think that much of the health care fallout is an expression of economic discontent, but there's certainly an argument to be made that it has exacerbated the public's predictable turn away from liberalism. In either case, however, disentangling these factors is extremely difficult.
Regular readers may recall my personal pet peeve about rushing to quick conclusions about the "most accurate pollster" in any given election. One of my objections -- all votes are typically not counted immediately -- is slightly less of a worry when applied to yesterday's Massachusetts race, as election officials there have produced an "unofficial" count based on 100% of the state's precincts. Still, the final certified count can sometimes differ slightly, sometimes enough to move the final count by a percentage point, so please take what follows as preliminary.
Another objection of mine, however, is even more valid in looking at a single state as it was back in February where we had several contests to consider:
[T[he whole notion of crowing a "big winner" based on a handful of polls in a handful of states is foolish. The final polls yesterday had random sampling error of at least +/- 3 percentage points. If a poll produces a forecast outside its margin of error, that's important. But if several polls capture the actual result within their standard error, chance alone is as likely as anything else to determine which one "nails it" and which miss by a point or two.
Let's use today's results to illustrate the problem. The following chart shows Brown's percentage of the vote as reported by the public polls conducted during the last 7 days of the campaign, with an error bar based the poll's reported margin of error. The horizontal bar represents Scott Brown's actual percentage of the vote.
What stands out most is that most of the polls produced an estimate of Brown's percentage of the vote within their own margin of error of the actual result. The exceptions are those on the left, which were conducted almost a week before the election, and if you followed our chart or read Charles Franklin's post on Monday, you know that Brown's support rocketed up over the course of January, so we should expect some of the earlier polls to show Brown's support lower than it turned out to be.
Another perennial issue with measuring the accuracy of pre-election trial heat questions is the issue of how to handle the undecided percentage. I did not allocate undecideds, and some polls had a bigger undecided percentage than others. Also, in this case, some pollsters included independent Joe Kennedy as an option, others did not. Kennedy ultimately received only 1% of the vote, but the Blue Mass Group/Research 2000 poll that missed the Brown percentage by 11 points had the biggest percentage either for Kennedy (5%) or undecided (5%).
So by and large, it is fair to say that all of the surveys conducted after Wednesday produced estimates of Brown's vote that were as accurate as a survey can be given the potential for random error.
You might reach a different conclusion, however, when you look at how they did forecasting Martha Coakley's percentage.
Here four surveys, all conducted after Wednesday, all significantly understated Coakley's percentage of the vote. Two were conducted for Pajamas Media by the Republican firm CrossTarget. The others were done by InsiderAdvantage and and by the Merriman River Group for InsideMedford.com. These four surveys are mostly responsible for the small understatement of Coakley's support in our overall trend estimate.
For what it's worth, all four used an automated IVR methodology and were completed in a single day, but all four also reported slightly higher undecided percentages than the others surveys conducted over the final weekend. So perhaps their significantly lower estimates of Coakley's support had something to do with their calling procedures, or perhaps they were not pushing undecideds hard enough.
I also produced the following table that calculates the error on each poll for each candidate and the error on the margin. The two surveys that missed the Brown's margin by the most were the Pajamas/CrossTarget and BlueMassGroup/Research 2000 polls conducted more than four days before the election - and they managed to miss in opposite directions.
Talk Business Quarterly / The Markham Group (D) / The Political Firm (R) / Wilson Research Strategies (R)
1/13-15/10; 600 likely voters, 4% margin of error
Mode: Live telephone interviews
Pew Research Center
1/14-17/10; 1,003 adults, 4% margin of error
Mode: Live telephone interviews
All in all, do you approve or disapprove of the way the Obama Administration has handled the
government's response to the situation in Haiti?
64% Approve, 14% Disapprove
From what you've seen and heard, do you think the U.S. government is doing too much, too little, or the right amount in providing assistance to Haiti?
11% Too much, 16% Too little, 58% Right amount
In general, how would you rate the job the press has done in covering the earthquake in Haiti?
81% Excellent/Good, 17% Only Fair/Poor
Do you think news organizations are giving too much, too little, or the right amount of coverage to the earthquake in Haiti?
19% Too much, 10% Too little, 69% Right amount
Most Closely Followed Story
57% A major earthquake in Haiti
18% Debate over health care reform
11% Reports about the condition of the U.S. economy
3% The current situation and event in Iraq
1% NBC rearrangomg its late-night comedy programs with Jay Leno and Conan O'Brien
As the federal government gears up for its decennial count of the country's population, most Americans think the census is very important and say they will definitely participate. But acceptance of and enthusiasm for the census are not universal. Certain
segments of the population such as younger people, Hispanics and the less well educated are not as familiar with the census and are less inclined to participate. In addition, there are partisan differences in opinions about the value of the census, and in personal willingness to participate.
The survey finds that nine-in-ten Americans describe the census as either very (60%) or somewhat (30%) important for the country, and about eight-in-ten say they will either definitely (58%) or probably (23%) participate. But 8% describe the census as unimportant for the country, and twice that number says that they either "might or might not" participate (10%) or definitely or probably will not (6%). The share saying they may not participate is particularly high among younger Americans, as well as those in lower socioeconomic categories.
Yesterday, Scott Brown became the first Massachusetts Republican elected to the U.S. Senate since Ed Brooke in 1972. Here is what happened:
This was not all about healthcare reform. Celinda Lake (Coakley's pollster) was right--to a certain extent--about healthcare reform's effect on the race. The bottom started falling out for Coakley when the Senate passed its healthcare bill in mid-December. But that isn't the sole explanation for Coakley's collapse or even the primary reason. A strong candidate with a better message could have withstood the national climate--especially in Massachusetts.
Candidates matter. Coakley was an awful candidate. As one KSDK reporter in Boston said: "She looked like she would rather eat broken glass than shake hands with a voter." Her "Rose Garden strategy" was a complete failure since it prevented her from engaging an angry electorate.
Coakley seemed disconnected. The cumulative effect of her gaffes (Schilling, no terrorists in Afghanistan) suggested to voters that she really didn't "get it."
But that doesn't mean that Democrats shouldn't be worried. Yes, she was a bad candidate and the country is in an anti-incumbent state of mind. That said, we're still talking about Massachusetts here. Obama was elected to bring about change, but voters--starting with the stimulus and continuing with healthcare reform--are anxious and frustrated about the policies that the Obama Administration and Democrats in Congress have pursued. It started in NJ and VA: this is the continuation of that trend, and it points to huge problems for Democrats in 2010, especially if they misread the tea leaves and try and push through a healthcare bill that the public has decided it doesn't want.
The country is still center-right. This election, along with results from New Jersey and Virginia, confirms what we--and plenty of others--said after the 2008 election: this remains a center-right country. Democrats should recognize this as they consider how to move forward on healthcare reform and other controversial issues (cap and trade, etc.).
The Coakley team completely misread the environment. They ran Coakley as the "heir apparent" to Kennedy. This was an anti-establishment/anti-incumbent election. It was like watching Mark Penn run the failed Hillary Clinton campaign all over again.
Going negative in the final three-day weekend was a huge tactical mistake by the Coakley campaign. The ads were too harsh and it seemed to have a galvanizing effect for Brown and his campaign. The Coakley team probably looked at internal polling last week and deemed it necessary - but it was still a mistake. Look, it is sometimes acceptable to go negative (use comparative spots) and paint your opponent early in the race, but only after you have laid out a positive foundation for yourself. Coakley never did that, so the ads seemed desperate.
Which brings us to the biggest problem: the Coakley campaign was asleep at the wheel. They never defined their candidate--or her opponent--in a meaningful and relevant way. They never framed the election in their favor. They never engaged the electorate on issues that mattered. Brown did all of the above. That's why he won.
Brown was the right candidate at the right time. If this election had taken place in early December or early February, we might have seen a very different result.
Deval Patrick was also a millstone around Coakley's neck. The Governor is in trouble with Massachusetts voters and it hurt Coakley.
State Democrats should not have rigged the succession system--twice. We wouldn't even be analyzing why Coakley lost if Democrats hadn't twice changed the Massachusetts rules for filling an empty Senate seat.
Independents carried the day for Brown. There are simply not enough Republicans in the state to give Brown a 100,000-vote victory. Turnout was high (50%), as 2.25 million voters went to the polls. Without exit polls it is hard to know for sure, but Brown had to win independent voters by a 2 to 1 margin. Chuck Todd is right: Brown, like McDonnell in VA and Obama in 2008, won by winning the middle.
Obama probably helped keep it closer than it would have been. Our sense is that because this thing really caught fire last Friday and Saturday, the media attention and Obama's visit actually helped stem the Brown surge by energizing some lethargic Democrats. Without it, Brown was trending toward a 9-point victory. That is why on Monday we tweeted a projection of 52% to 47%. Late votes went to Coakley. But there weren't enough of them.
This should not have been a complete shock. Coakley's highest vote share in pre-election polls was 58% in November. In Massachusetts, she should have started out with 63-65%. She was NEVER in a strong position and was ripe to be picked off if the national mood soured. And it did.
Massachusetts has never elected a woman as U.S. Senator or Governor. Gender was probably not a major factor but some of Brown's internal polling suggest that voters thought Coakley would be weak on national defense/terror issues. Part of this is due to the administration's poor reaction to the Christmas Day bomber and broader concerns about Obama's performance on national security issues, but gender may also have had a role.
Thanks to Pete Ventimiglia and John Zirinsky for their insights and analysis. Please follow us on Twitter for up-to-the-minute analysis.
A round-up of Pollster-centric reactions to the Massachusetts results:
This Boston Herald lead speaks volumes: "High turnout in Bay State 'burbs and among independent voters who flocked to the polls eclipsed a healthy turnout in staunchly Democratic Boston, fueling Republican Scott Brown's victory yesterday."
The question of the moment is what effect Scott Brown's victory will have on national politics.
It's important to note that his election to the Senate does relatively little to change the overall balance of power in the country. See, for instance, Joshua Tucker's helpful chart:
The loss of Democrats' filibuster-proof majority seems to eliminate the prospect of passing the health care bill through conference committee, but for other legislation, the shift of the pivotal voter from Ben Nelson to Olympia Snowe in the Senate is likely to have a relatively small direct effect. Nelson is currently paying a heavy political price in Nebraska for his support of the health care bill and is unlikely to take a similar risk on future legislation. (On a more technical level, Tucker also notes that the gap between Nelson and Snowe's ideal points is probably relatively small -- see, for instance, Simon Jackman's estimates [PDF].)
Similarly, we knew Democrats faced an unfavorable environment two weeks ago and that the health care reform plan was relatively unpopular in national polls. Not much has changed on either front.
What matters, however, is the collective interpretation of the election. Even though Brown's victory was an ambiguous amalgam of national and local factors, including Coakley's hapless campaign and poor economic conditions, the media is already portraying the outcome as a referendum on President Obama (though a majority of Massachusetts voters approve of his performance) and health care (even though Brown supports a very similar state-run plan in Massachusetts). Debatable as they may be, these interpretations may quickly become conventional wisdom -- indeed, many Democrats have already endorsed them.
The most relevant comparison to the current situation might be electoral mandates. The seminal political science research on the subject shows that opposition party legislators tend to deviate from their typical voting patterns in the direction of a perceived mandate for some period of time before returning to normal.
Given the Democratic tendency to panic in these types of situations, we may see a similar shift in voting patterns or a change in the party's legislative agenda. Pundits will likely claim that Democrats should yield to public opinion as expressed by Massachusetts voters. But it's not at all clear that such moves will prevent significant losses in the November midterms, nor that there is a "message" from Brown's victory as such.
Update 1/20 1:50 PM: Based on Brown's voting record as a state legislator, political scientist Boris Shor estimates that he will become the Senate filibuster pivot rather than Snowe. As I've previously argued, I think Brown moved right to motivate the GOP base in a low-turnout special election, so I'm skeptical he'll pursue such a moderate course (at least right away). But if Shor is correct and Brown is between Nelson and Snowe, it reduces the rightward shift in the filibuster pivot, meaning that Brown's win would have an even smaller effect than we might have otherwise thought.
Update 1/21 9:36 AM: See also John Sides on the need to admit what we don't know about the MA results and Greg Marx on the media's misguided attempts to distill a "message" from the election.
Update 1/22 9:44 AM: Via Matthew Yglesias, Alec MacGillis reports in the Washington Post that "Brown's victory in Mass. senate race hardly a repudiation of health reform."
The distribution of the vote shifted for Martha Coakley but not for Scott Brown last night. That was the key to Brown's win.
The top left of the chart above shows the distribution of Coakley's vote compared to Brown's. Brown's better total shifts his distribution clearly to the right. That's not interesting. But the bottom row is very interesting. The bottom left panel compares Coakley (dark blue) with Obama's light blue distribution in 2008. She's well to the left, doing worse. Of course you'd expect drop-off from a presidential to a special election. But the bottom right panel is amazing. Brown's distribution almost exactly duplicates McCain's. In a January special election, Brown's vote is a clone of McCain's in a presidential contest. That is amazing.
Here is another way to look at it. Plot last night's vote by town against their party's candidate in 2008.
Brown's votes are almost exactly on the 45-degree line showing equality between 2008 and 2010 vote totals by town. But not so the blue dots, which are all, yes every single one, well below the diagonal. Brown's total actually slightly improved upon McCain's. Coakley's total was just 56% of Obama's total.
The chart is powerful but the logarithmic scale makes the two clusters of points appear closer than they "really" are. Let's plot Coakley as a percent of Obama vote against Brown as a percent of McCain for a more compelling view.
Wow. The imbalance of performance is stark. Coakley's BEST town gave her 80% of Obama's vote. That's equal to Brown's WORST towns. Even the towns Coakley won were places she was dramatically underperforming Obama. And there are no pockets of strength visible here. Brown was doing over 100% except in a few blue towns but he even outperformed McCain in a number of towns that went for Coakley.
Of course this doesn't mean that Brown got exactly McCain's voters, since lots of individual switching could add up to these totals. But in the aggregate, Mass. in 2010 looks exactly like it did in 2008 on the Rep side. On the Dem side, a whole lot fewer voters.
We liveblogged these last night. Compulsion makes me post the final versions with 100% of precincts reported. Do note that it is common for the final certified vote to differ a bit from these 100% election night totals, so the final percentages and margin may move a tad.
Above is the trend in Brown's percent of the vote over the evening. Stable, but with a downward trend as the more Democratic areas reported relatively late in the evening after about 70% of the vote was in.
Below is the vote margin by estimated vote outstanding. It wasn't close enough to matter last night, but what you like to watch for is when the margin is getting big relative to the outstanding vote. At some point it becomes mathematically impossible to close the gap. That happened last night with 95.3% of precincts reported. Then the fat lady sings. (By the way, you read this one from right to left.)
What is your preference for the outcome of this year's congressional elections--a Congress controlled by Republicans or a Congress controlled by Democrats?
41% Republican, 41% Democratic
In the next election for U.S. Congress, do you feel that your representative deserves to be reelected, or do you think that it is time to give a new person a chance?
39% Deserves to be re-elected, 49% Give new person a chance
From what you have heard about Barack Obama's health care plan, do you think his plan is a good idea or a bad idea?
33% Good idea, 46% Bad idea (chart)
27% Democrat, 19% Republican, 46% independent (chart)
When I moved from DC to Amherst in August I was looking forward to the charm of a small New England college town and the relative affordability of housing (compared to prices inside the beltway, at least). But what I knew I'd miss the most was living at the center of the political universe. Well, for one day at least, I get to re-live the excitement as all eyes turn to the Bay State.
Warning: what follows is entirely un-scientific and is, accordingly, of little use to understand what might happen when the polls close tonight.
I've been surprised over the past few weeks at how much of a ground game Brown seems to have in Western Massachusetts compared to Coakely. Last weekend, I had to drive to nearby Belchertown for a swim meet and we passed countless yard signs for Brown as well as a small rally of Brown supporters. Not a single sign for Coakley. Today I had to travel to Boston for an appointment and I was 40 minutes into the drive before I saw my first Coakley sign.
Of course, once I got into Boston things changed quite a bit. Coakley signs were much more prominent and the handful of polling sites I passed were packed. This is consistent with what the news has been reporting regarding high turnout. But what was most interesting from my vantage point (and the vantage point of any pollster trying to determine who will and will not vote today) was a conversation I had with a young Democratic store clerk who, upon finding out that I was a political science professor, started complaining about how he ended up having to get up early today to go vote when he had been planning all along to skip this election. I've had similar conversations with a number of Democrats over the past few days; people who had no intention of voting a week ago, but now feel compelled to do so. These people are not at all excited about the Coakley campaign, but they suddenly feel as though they have to go out to vote in an election they were planning on skipping.
PS: As I noted a few days ago on Twitter, the networks are not be conducting an exit poll in Massachusetts today. Network interest in doing one, I'm told, occurred too late for the long lead time necessary. We'll probably live-blog again tonight...check back in later for details.
Generally speaking, do you strongly favor, somewhat favor, somewhat oppose or strongly oppose the health care reform plan proposed by President Obama and the congressional Democrats?
38% Strongly/Somewhat Favor, 56% Strongly/Somewhat Oppose (chart)
My column today considers the proliferation of inexpensive automated surveys and two developments that promise to fuel even more growth: Two companies that promise to offer the ability to conduct an automated telephone poll for less than a thousand dollars.
I filed the column on Friday afternoon, which in Massachusetts Senate time was seven polls ago, and those new polls underscore the message of the column. Nine of the 16 polls we logged on Pollster.com since January 1used an automated telephone methodology, including 5 of the last 7. Two of the surveys, those sponsored by Pajamas Media and and InsideMedford.com, used pollsters whose work we have not previously tracked.
Of those 15 polls, only two -- the surveys conducted by the University of New Hampshire and Suffolk University -- had traditional, mainstream media sponsors. The rest were conducted or sponsored by polling public relations, political partisans, "new media" web sites or some combination of these three.
So the vision of the future described in the column is, in many ways, already upon us.
Although we're still nine (9) months away from the 2010 elections, DC elites are already handicapping the outcome.
In order to get a better sense of what official and political Washington is thinking about the 2010 elections, StrategyOne appended a question on our recent Beltway Barometer survey. The full (crosstab) results can be found here.
So what are the DC insiders thinking?
First, both Democrats and Republicans think Democrats will lose seats in the House in 2010. Only 7% of Democratic elites think Nancy Pelosi will maintain (5%) or increase (2%) her seat margin in the House. The question Washington elites are pondering is not IF Democrats will lose seats in the House, but HOW MANY seats they will lose. This HOW MANY question is where DC elites differ.
Elite Republicans in Washington generally expect to see their party gain between 20 and 39 seats in November. 56% of elite Republicans expect Democrats to lose between 20 and 39 seats, and 25% of elite Republicans expect Democrats to lose 40 or more seats and with this their majority. It is interesting to note here that Charlie Cook's current forecast (Democrats lose 20-30 seats) tracks closely to majority elite Republican sentiment. It is also interesting to compare this survey data to National Journal's Congressional Insiders Survey.
At this stage elite Democrats in Washington expect to keep the losses under 20 seats. Specifically, 62% believe Democrats will lose fewer than 20 seats and 28% believe Democrats will lose between 20 and 39 seats. Only 2% of elite Beltway Democrats currently think that Democrats will lose the House in November. This sentiment may change if Coakley loses today.
StrategyOne will be tracking elite DC opinion on the 2010 elections and reporting this data first to our clients and then to the media (and pollster.com readers).
The warning signs are certainly present for Democrats at this point. I noted in my December 16th article (What Does Bart Gordon's Retirement Tell Us?) that the Moore, Tanner, Baird and Gordon retirements certainly appear to be warning signs reminiscent of past tough election cycles. Vic Snyder's (AR-2) retirement announcement last week only seems to add to this fact pattern.
Although wild card events could easily change the arc of the 2010 election season, the trend is certainly beginning to suggest a Republican wave.
I know it's a long way off, but if David Paterson runs for Governor in 2010, would you vote to elect him or would you prefer someone else?
21% Elect Paterson, 60% Prefer someone else
I know it's a ways off, but if Kirsten Gillibrand runs for United States Senator in 2010, would you vote to elect her or would you prefer someone else?
29% Elect Gillibrand, 45% Prefer someone else
Would you prefer to see Attorney General Andrew Cuomo run for re-election as Attorney General this year or would you prefer to see him run for Governor instead?
30% Re-elect as Attorney General, 53% Run for Governor
ABC News / Washington Post
1/12-15/10; 1,083; 3.5% margin of error
Mode: Live telephone interviews
(ABC: story, results; Post: results)
Overall, given what you know about them, would you say you
support or oppose the proposed changes to the health care system being developed by Congress and the Obama administration?
44% Support, 51% Oppose (chart)
The plan could include a government insurance option only for people who can't get private insurance, the so-called public option. Or instead it could have the government negotiate with private insurers to have them offer these people insurance that meets government specifications. Which of these two approaches would you prefer - the public option or the alternative?
47% Public option, 41% Alternative
The plan could include a tax on high-benefit health plans, or a higher income tax on wealthier Americans. Which of these two approaches would you prefer?
22% Tax on high benefit health plans, 58& Higher income tax on wealther Americans
Insurance plans in which the government is involved could be forbidden from covering abortions. Or they could have an option for people to buy private coverage for abortions separately through these plans. Which of these two approaches would you prefer - the one that's more restrictive on abortion coverage or the one that's less so?
50% More restrictive, 48% Less restrictive
Say a candidate for Congress supports the proposed changes to the health care system being developed by Congress and the Obama administration. Would that make you more likely to support that candidate for Congress, more likely to oppose that candidate, or wouldn't it make much difference in your vote?
22% Support, 31% Oppose, 45% No difference
Over the last few days, thousands of words have been written and many charts posted on Pollster and elsewhere, all trying to make sense of the sometimes divergent Massachusetts Senate polls. If you haven't yet, make sure you read Charles Franklin's tour de force review posted here a few hours ago, that walks through eighteen permutations of trend line models, all of which show Republican Scott Brown leading Martha Coakley, most showing him ahead by 4 points or more.
If you prefer a simpler summary, consider this:
Of eight surveys completed and released since Wednesday, seven show Brown leading by at least a point. The one exception shows a dead heat Our chart of all polls shows a nearly seven point Brown gap between the trend lines for Brown and Coakley (51.2% to 44.3%).
Browns' support on our standard trend estimate has increased by nearly twelve points (from 38.5% to 51.2%) in just the last two weeks.
A trend this strong is unusual, especially in a contest between a Democrat and a Republican. We do see such surges occasionally in primary elections -- the surprise victory by Creigh Deeds in last year's surprise victory in Virginia's Democratic primary being the most recent example -- but they are far more rare in general election contests . Over the weekend, I reviewed the most competitive contests we have tracked on Pollster.com since 2006 and found no race that produced a trend anywhere near this strong over the last few weeks of the campaign.
I am sure that there are other example, but the one that stands out for me is the victory of Democrat Harris Wofford in 1991. Wofford, appointed earlier that year to fill a vacant Senate seat, began as a virtual unknown and began trailing by more than 40 points against popular former Republican Governor Dick Thornburgh. Although the final round of public polls showed the candidates running about even, Wofford's momentum helped carry him to what turned out to be an eleven point victory margin (55 percent to 44 percent).
Of course, the same factors that make the trend toward Scott Brown so unusual also make the polling challenging and potentially misleading. Brown has moved up so rapidly partly because campaign has been truncated, but the rapid change also prompted a late avalanche of negative advertising by the Democrats directed at Brown. Because it is a special election being held on an usual date, Pollsters have no prior history to judge the size and demographics of the likely electorate. The likely voter problem is one reason why polling errors tend to be larger in special elections.
So while we have the Wofford experience on one hand, we have the lessons of the New Hampshire Democratic primary in 2008 and the special election in New York's 23rd District this past fall on the other. In both cases, candidates surged in the final polls, only to see their apparent leads disintegrate on Election Day. What those races had in common were huge surprise developments that occurred a few days days before the election (Barack Obama's Iowa victory and the withdrawal of Dede Scozzafava) that helped shake up the race, fuel the polling surge and -- perhaps -- provoke voters to focus more closely on their choices and rethink their preferences in the final hours.
Does the nationalization of the Massachusetts Senate race combined with the heavy negative advertising blitz qualify as the same sort of last minute surprise? Perhaps, but it seems like a stretch to me.
Some believe that non-response bias may have contributed to the errors in those two races, exaggerating the contribution of the most enthusiastic supporters of the surging candidate. Mike Mokrzycki developed that theory in the context of the Massachusetts Senate race here over the weekend. Some believe this phenomenon may be more acute in automated surveys, and we should not ignore that only two of the last seven public polls used live interviewers.
Yes, Coakley has done better on live interviewer surveys than the automated polls, but we saw a similar pattern in New Jersey last fall, and the robo-polls ultimately provided a closer forecast of the final margin.
Yes, the internal Coakley campaign poll numbers that have leaked out show a dead even race and perhaps a slight improvement over last week. However, there was more than one internal poll conducted by Democrats A little birdie tells me that the final tracking survey conducted by the Mellman Group for the Democratic Senatorial Campaign Committee had Brown ahead by five points.
So for me it boils down to this: I was a Democratic consultant for long enough to want to believe that Coakley can still prevail, and there is still a remote chance that the polls in this race will be as misleading as they were in New Hampshire. However, my head is not my heart. Barring another polling meltdown, Scott Brown is the likely winner.
This is the first of two posts that will wrap up my thoughts about the Massachusetts Senate race. This first one covers an admittedly narrow point. It is provoked by an email I received reacting to Nate Silver's post from Sunday -- "A Statistical Ray of Hope for Coakley" -- from Pollster reader Harry Enten (aka commenter Poughies):
[Silver's] piece makes the point that the margin between Martha Coakley and Steve Brown could be overstating Brown's lead. Silver points out that polling in close (margins of 10 or less in the polls) Senate elections since 2000 in deeply blue states (as measured by the Cook Political Partisan Index) has by an average of 3.4 points underestimated Democratic candidates' margin of victories. In deeply red states, on the other hand, polling has underestimated Republican margins by 1.9 points.
Though he used a Pollster.com average in 2008 and Real Clear Politics average in 2006 and 2004, Silver used a simple average of all non-partisan polls conducted in the final two weeks compiled by Pollingreport.com in 2002 and 2000. Perhaps, he did not know, but a Real Clear Politics average is also available for 2002 and 2000.
I was interested what, if any, effect substituting the Real Clear Politics averages in 2002 and 2000 would have on Silver's results. Therefore, I decided to create a new dataset modeled after Silver's, but using the Pollster.com data in 2008 and Real Clear Politics average* in 2006, 2004, 2002, and 2000.
Using these new rules, the underestimation of Democratic margins in blue states stays the same at 3.4 points. This result is not surprising, as the blue states part of the dataset is small, and only three results are available from 2000-2002. The underestimation of Republican margins in red states drops from 1.9 to 0.9 points.
The overall average of underestimation drops from 2.3 to 1.5 points. In only 7 of 23 contests was the underestimation above 3 points. In no contest did the polling average incorrectly predict the winner due to underestimation of Democratic candidates in blue states or Republican candidates in red states. The only incorrect winner chosen was in the South Dakota (a red state) 2002 race when the polls predicted a victory by Republican John Thune.
The bottom line is that perhaps the polls are overestimating Brown's margin in Massachusetts. The limited data involving closely polled elections in blue states suggest that Coakley might do better than the polls suggest, but when you look at the larger dataset of red states, Coakley should not expect a bump.
*In some cases (such as Alaska 2004), no Real Clear Politics "average" existed. I just averaged all the polls listed on Real Clear Politics (and in the case of Louisiana 2002) conducted in the final two weeks. I, unlike, Silver use internal polls in these cases... modeling myself after Pollster.com's inclusion of them.
P.S. The Real Clear Politics web pages can be hard to find [but are available at these links] for 2002 and 2000.
He also passed along a table (in Excel and PDF formats) with the numbers.
It's also worth taking a closer look at the six races -- there were only six -- that are the basis for the conclusion that polls understate support for Democrats:
Most interesting are the three races (Maryland and Rhode Island in 2006 and New York in 2000) that produced the biggest errors. Of these, the Rhode Island example is partly the result of an RCP average based on just two surveys conducted over the final weekend of the campaign. We used a simple last-5-poll average for Rhode Island that year, which showed Whitehouse winning by six, just one point off the actual margin. Just swapping the Pollster and RCP averages for that one race would cut the average variance to +2.5. Whatever the challenges of polling in Massachusetts, greater random error due to a shortage of final polls is not one of them.
That leaves two big errors affecting Ben Cardin in Maryland in 2006 and Hillary Clinton in New York in 2000. What's interesting about the errors in both contests was that the final round of polls had the percentage for the Republican candidate about right (within a point), but understated the support for the Democrat by about five percentage points. Even if we assume that a similar pattern will apply in Massachusetts tomorrow, the problem is that the six of the last seven surveys estimate Brown's support at 51% or greater.
Republican Scott Brown holds a lead in all 18 alternative models of the Massachusetts Senate race polls, now including all polls released through 6:00 p.m. Monday. Our standard trend estimate puts the race at a 6.2 point Brown lead over Democrat Martha Coakley. The less sensitive alternative linear model puts the Brown lead at 7.3 points. Across all models, Brown leads by between 1.0 and 8.9 points. Three quarters of the estimates have Brown ahead by 4 points or more.
Brown built this lead over the past week of polling with only some tentative sign of the trend flattening over the weekend. Of course the last available polls were completed Sunday evening so we do not know if any movement has occurred on Monday.
Here is a brief review of the polls and the various models estimated. First, the polls without any trend estimates:
One of the unusual features of the MA polls is the large number of leaks from Coakley's internal polling. No one leaks without a reason, and her leaks have been consistently better for her than other polling taken at the same times (with one exception). Past analysis has found that internal polls are typically about 5 points better on the margin for the leaker than are independent polls, but that the internal polls do track the trend rather well. This raises a question of how to treat Coakley's polls. Below, I estimate the models both with and without the leaks included, so we can see their potential impact. I have not discounted them for the historical five point bias with internal polls.
First, let's estimate the local regression models that are our standard here at Pollster. These are not identical to the dynamic charts because here I am estimating the Dem minus Rep margin while our charts estimate each candidate separately. I estimate the standard model, a more sensitive and a less sensitive version, and then repeat with the leaked polls included.
With the limited number of polls, the local its are not as smooth as in our usual trends with dozens or hundreds of polls. They may also be more sensitive to outliers. That is one reason to check the effect of sensitivity. For the three models without leaked polls, the sensitivity matters a bit for the trajectory but hardly at all for the endpoints. When internal polls are included, the trends end up a couple of points more Democratic, though still put Brown ahead in the end. (The more sensitive estimate touches dead even but that is due to a day with only an internal poll. The sensitive estimator chases that but then moves back down.)
Next I switch to even less sensitive linear models. The local trends show some non-linear movement, and suggest a small upturn in the last day of polling, which the linear models will miss. But with relatively few polls, much of the "bendiness" of the local trends is due to noise and overfitting the data rather than meaningful shifts. The linear fits are a hedge against the noise, at the expense of an ability to spot a reversal of trend.
With these models we can disaggregate the data by partisan affiliation of pollsters. This gives a range of estimates from most favorable to Republicans to neutral (no picking of polls) to most favorable to the Democrats. Here I include the leaked polls in the most Democratic model, using only Dem polls plus the leaked internals, and in one model with everything we have regardless of source. The result is a range of estimates. The most Democratic model shows a 1 point Brown lead. Others range from about 5 to about 9 point Republican margins.
A final variation in the models is to fit quadratic models to allow the trends to bend according to how much the data demand a bend. If there were substantial upturns (or downturns) at the end, the quadratic model could pick that up while still maintaining less sensitivity than the local regressions we started with.
As it happens, the data don't demand much bend at all. The fit is only slightly better with the bend and the end points of the lines are only modestly changed.
Finally, let's see the garbage can of all the models at once, to see if any stand out as very different.
There is a range of estimates, but all are below zero, indicating a Republican lead. The two most Democratic models (using only Dem polls plus the internal leaks) stand out as most different from the rest. Half of the models fall between -4.3 and -7.4. The two most Republican estimates put Brown's lead at about 8.8 points.
The caveats are that turnout may yet matter, for either side. Reps enjoy an enthusiasm advantage, according to the polls, but Dems might yet mobilize their voters beyond what the polling suggests. And there is the unknown of the GOTV efforts on Tuesday. But if Coakley wins, this will be a major surprise, and the pollsters will have a lot to rethink about their methods. A win for Brown will have huge implications for the Democratic policy agenda and will put the fear of God into Democrats running in November.
There has been a wider than normal range of polling results in the last two weeks from the Massachusetts Senate special election. This has been further clouded by a number of leaked internal polls and polling by relatively unknown and unproven pollsters, some partisan but others not. And most importantly, the rapid shifts in the race, reflected across all the polls, makes this a fast moving target. So let's take a moment to consider what we could reasonably conclude based on the data.
But no matter how you slice the data, the only reasonable conclusion is that Scott Brown has moved from well behind to a lead somewhere between 4 and 11 points.
The chart above shows all the polls we have available as of 12:36 a.m. Monday morning. That includes new PPP and Pajamas Media/CrossTarget polls released late Sunday evening. The chart also includes the leaked polls, mostly from the Coakley campaign but one from Brown as well. These leaked polls are NOT included in most of the estimates above, though they are not out of line with the rest of the data.
So what might you believe about these data? You could refuse to cherry pick the polls. That has long been our view here at Pollster.com. Our job is to summarize the trends as best we can, without partisan favor. If you do that, we get a 8.8 point Brown lead.
Perhaps you only trust non-partisan polls. Then the Brown lead is 6.8 points.
Maybe you are a Dem, who doesn't trust the Republican pollsters. Then Brown leads by 6.5 points.
Or you are a Dem who doesn't trust the non-partisan pollsters either and who does believe in the leaks from the Coakley campaign. Then Brown's lead is 3.8 points. (This is the only estimate that includes the leaks.)
Or you are a Rep who trusts GOP and nonpartisan polls only. Then Brown leads by 11.3. (There aren't enough Rep polls to run a Rep only estimate to parallel the Dem only, but I'd think an 11 point lead would be satisfying enough for Reps.)
There may be other ways to cut these data (IVR vs conventional phone, pollsters you've heard of vs ones you haven't) but it seems quite unlikely that any but the most selective reading of these data can find that the race remains a dead heat. Brown has a lead, as of Sunday night.
Let's back up a step to look at the data without the clutter. Here are just the polls, no trends fit.
Without the lines it is quite clear that the movement has been sharply towards Brown. Trace out what you like, ignore what you don't like, in the early polls Coakley is convincingly ahead. Then between about day -8 and -5 the polls are balanced above and below dead even. Since then no poll has shown a Coakley.
In my models in the first chart, I use linear fits rather than our usual local regressions. The reason is there are still not very many polls,and once we subset them by party there simply aren't enough cases to get good local regression fits. That subsetting is the main point here. But it also turns out that the local regression on all the data isn't very far from the linear fits I use above. Here is the comparison:
The blue trend is our standard estimate, and it wiggles a bit due to only 12 cases. If we use a bit less sensitive local regression, we get the black line. And the red linear fit isn't very far from either of the two local fits. So I'm willing to give up some flexibility in the fit for a bit more robustness, and especially the ability to fit the models by party of pollster that was the lede above.
Finally but significantly, we are seeing more pollster variation in this race than normal. If we look at the residuals around the trend estimates, past experience with 2004, 2006 and 2008 state and national contests has pretty consistently found that most of the polls (about 95%) fall within +/- 5 points of the trend estimate. Now that is an empirical observation, not a theoretical one. But it has been generally consistent in our data. How do these polls compare?
Only half of the current polls are inside +/-5 points of the linear trends. The number of polls is small, and this race is more dynamic than most. But one has to wonder about the problems of polling in a special election, the role of partisan and new players in the polling and the heavy use of IVR polls. This is much more variation in polls than we normally see in general elections.
Let's also recall the NY-23 special election, which was not polling's finest hour. The last three polls there had Hoffman up by 5, 5 and 17 points. Our final trend estimate based on all the polls had Hoffman up by 5, 41.8 to 36.8.
Polling special elections is hard. Tuesday we'll see how hard, and who was good and/or lucky.
Mike Mokrzycki is an independent consultant who was the founding director of the Associated Press polling
unit. He may be reached at firstname.lastname@example.org. His guest contribution is cross-posted from his blog, MJM Survey Musings.
One thing is certain about the polling in the last days before
Tuesday's special election in Massachusetts to fill the late Ted
Kennedy's U.S. Senate seat: Someone's going to end up being very, very
Polls completed in the past week and recorded at Pollster.com
range from a 14-point lead for Democrat Martha Coakley - just weeks ago
considered a shoo-in in heavily Democratic Massachusetts - to a
15-point advantage for Republican Scott Brown, who has become a darling
and major fund-raising beneficiary of conservatives nationwide.
I'm not going to do a deep methodological dive into all these polls to try to explain the differences. Pollster.com and Fivethirtyeight.com
have done their usual stellar job with that already, including
analyzing the extraordinary uncertainty inherent in trying to determine
who really will vote in this mid-January special election.
I will try to provide a little perspective as someone on the ground in Massachusetts who also knows a thing or two about polls.
My hypothesis: While Brown supporters clearly are more enthusiastic
than Coakley backers, that may serve him relatively better in the
pre-election telephone polls than it will Tuesday.
I've lived in Massachusetts on and off since 1980 and I can't ever
recall Republicans here as energized as they are now. Sure, they had a
16-year run in the governor's office despite the state's overall
leftward tilt. But Bill Weld, elected in 1990, was fairly unusual -
socially liberal enough that "Weld Republican" became its own label.
Paul Cellucci sure didn't inspire a lot of passion and I can't say Mitt
Romney did either, with his eye on the White House all along. Brown
seems an agile campaigner but I don't think his personal charisma is
what's charging up Republicans here and elsewhere; rather, it's the
once almost-unthinkable notion that any Republican might
actually win the seat Ted Kennedy held for nearly half a century,
especially with such extremely high stakes for policy and politics
This enthusiasm is abundantly evident in internal data from numerous
polls. I'd add a couple anecdotes: I don't put stock in lawn signs but
when you see a voter (like someone on the main street in my town)
posting a handmade Scott Brown placard, or an ice cream stand
using its roadside sign to advertise "VOTE FOR SCOTT BROWN," it may be
an indication something beyond rote partisanship is at work.
This race has been Coakley's to lose, and she's seemingly been doing
her best to do that. The most recent example was in a radio interview
the other day when she called Curt Schilling - famous for helping pitch
the Boston Red Sox to a long-awaited World Series championship in 2004
on an ankle stitched together and visibly bleeding through his sock - a
New York Yankees fan, of all things. In little more than the
time it used to take a Schilling fastball to reach the plate, his
recorded voice was on my phone telling me this faux pas was
proof Coakley was out of touch with Massachusetts voters. Silly,
weighed against the import of issues such as health care reform?
Perhaps. But - last baseball metaphor, I promise - Coakley served up a
big fat meatball and I sure don't blame the Brown campaign for hitting
it out of the park.
Schilling's was one of countless phone calls we've gotten on this
race since before the primaries last month. Many have been "robo-calls"
like his (as I write this paragraph I just got one from Brown's
daughter), though plenty feature live human beings (like someone from
Coakley's phone bank who called as I started writing this post).
At this point it's hard to blame people in Massachusetts for
screening incoming calls even more so than usual. For years there's
been plenty of screening, part of the reason why response rates for all
kinds of telephone polls have declined dramatically. (An article in the Winter 2009 issue of the journal Public Opinion Quarterly
(subcription required) gives response rates for numerous respected
telephone polls it cites and many of them barely crack 10 percent. A
response rate greater than 20 percent now is extraordinarily good.)
Response rates are even lower for automated polls, which use a recorded
voice for interviews and require respondents to punch in answers on the
But - and this is an important "but" - a growing body of research
indicates decreasing response rates have not hurt the accuracy of
survey estimates. That happens when there's no systematic difference
between those who cooperate and take the survey and those who decline.
I'm thinking the Massachusetts Senate race may be a case where we do
see non-response bias in surveys. It comes down to relative enthusiasm
for the candidates. It's tough to prove, but I'd venture a guess the
dynamic works like this:
Republicans are excited Brown might win and thus more likely to
answer their phone and listen to political messages -possibly be
invited to take a survey - when the phone is practically ringing off
the hook with such calls. I suspect they'd be particularly enthused to
participate in a poll and tell the world they're voting for Brown, to
help build the sense he has unstoppable momentum. These folks certainly
will vote but there's no upside in Brown's election-day numbers
compared to the pre-election poll estimates.
Democrats may be demoralized and scared after several weeks of
Coakley campaign missteps and bad headlines. They may not be all that
eager to pick up the phone for political calls. They also might be more
skeptical of or angry about polls since they've been such downers for
Coakley and President Obama lately, and thus, I would speculate, more
likely to take a pass if invited to participate in one. None of that
means these folks are less likely to vote, though - by now any sentient
Democratic-leaning voter will know Coakley needs all their votes, and
what's at stake. They might not be happy about how Coakley has run her
campaign but they'll still be motivated to vote by a desire to deny a
Republican the chance to do serious harm to Obama's agenda from Ted
Kennedy's old seat. Obama is in Massachusetts today to remind them of
exactly that (not that they're necessarily all that enthused about him
at this point, either).
Of course, truly independent or "swing" voters are another vital
factor, and if Brown wins enough of them he could overcome the inherent
Democratic advantage in Massachusetts. But I'd think enthusiasm, or
lack of it, would be more of an issue among stronger partisans.
In pollster speak, what this boils down to is "differential
non-response," where one candidate's supporters are more likely than
the other's to take a survey. It's suspected to be a big reason why
exit polls in recent years have tended to overstate support for
Democratic candidates. In the Massachusetts special Senate election I
suspect it's inflating the Republican's poll numbers. Coakley has room
to outperform the polls Tuesday even if her natural base is motivated
by nothing more than fear of what would happen if her opponent pulls
off an historic upset.
State of the Country
37% Right Direction, 62% Wrong Track (chart)
Would you say Obama is doing a better job as president than you expected, a worse job, or what?
33% Better, 35% Worse, 30% As expected
Obama has been president for about one year. Would you say he has accomplished a great deal during that time, a good amount, not very much or little or nothing?
47% Great deal/Good amount, 52% Not much/Nothing
Is Obama keeping most of his major campaign promises, or not?
41% Yes, 46% No
Changing topics, do you think Obama's economic program is making the economy better, making it worse or having no real effect?
35% Better, 23% Worse, 41% No effect
One of the odd aspects of last week of the Massachusetts Senate campaign is the way Coakley "internal" polling numbers have leaked on a near daily basis, through blogger Steve Kornacki and others. Wednesday night, according to Kornacki, the Coakley campaign's own polling showed her "barely ahead, 46 to 44 percent." Thursday night's results showed her trailing, 47 to 44 percent, and conservative columnist Byron York added a quote from an unnamed but "well-connected Democratic strategist" who "heard" that "in the last two days the bottom has fallen out of her poll numbers." Then on Friday night, again according to Kornacki, Brown was ahead by just two points on Coakley's poll (47% to 45%), and a three-day average of the results from Wednesday through Friday night gave Brown the same two-point lead (47% to 45%).
These leaks produced some snickering: Via Twitter, PPP's Tom Jensen pronounced the leaks the "sign of a highly undisciplined campaign." Jay Cost asked "how lame is the Coakley campaign" to leak their internal tracking polls "EVERY DAY?" And a very smart reader emailed this morning with the observation that leaks mark Coakley's campaign "more undisciplined than a 4 year old at K-mart on a sugar high."
Let me be clear: The conclusion that Coakley's campaign -- her staff or the consultants she retains -- is responsible for these leaks is probably unfair and a bit naive. They were likely not the source.
Now, alas, I do not have any inside information and have not been the recipient of any such leaks (really, old consultant comrades, where is the love?). But I can say from my own experience as a Democratic campaign pollster that it's fairly standard practice for a Senate campaign like Coakley's to share their daily tracking results with the Democratic Senatorial Campaign Committee (DSCC), the White House political office and EMILY's List. It's really unimaginable that Coakely, or any other "disciplined" campaign in their position, would not. Also, either directly or through these organizations, the same are almost certainly going to the labor groups and other interests conducting their own campaigns on Coakley's behalf. Each of those organizations has its own pollsters, media and direct mail consultants. So the leaks could have come from damn near any "well connected strategist" in Washington.
The incentive to leak would be especially high for those who have parachuted in to help in the final week, those with great incentive lay the groundwork to take credit should Coakley "come from behind" to win. Consider that these incentives are even greater for the White House. Obama really had no choice but to come to Massachusetts to Coakley's aid (as he is doing today). A Coakley loss will be catastrophic for Obama's legislative agenda, and the White House will take some of the blame either way. So a mid-week decision to come to her assistance creates huge incentive to leak these numbers. Again, if she loses, well, the bottom had already "fallen out." If she wins, they claim credit for turning things around.
Now all of this probably speaks to a breakdown in team play or the sort of ugly finger pointing that always seems to accompany defeat. For those surprised by the wide dissemination of "internal" Coakley polling data, consider that in the fall of 2008, the Obama campaign shared polling numbers and a whole lot more on a daily consultant conference calls whose participants (I'm told) close to a hundred. Nothing of significance leaked from those calls before Election Day. One way or another, Martha Coakely and her campaign are worthy of much criticism, but piling on over these leaks is unfair.
By the way, it's more than a little crazy to be paying much attention to the random zigs and zags apparent in the relatively small one-night samples used in internal campaign polling. I certainly hope that the pollsters of record are not making decisions or recommendations on the basis of anything but the three-night rolling averages.