Pollster.com

Articles and Analysis

 

Do Polls Underestimate Dems in Blue States?

Topics: 2010 , Harry Enten , Martha Coakley , Massachusetts , Nate Silver , Scott Brown

This is the first of two posts that will wrap up my thoughts about the Massachusetts Senate race. This first one covers an admittedly narrow point. It is provoked by an email I received reacting to Nate Silver's post from Sunday -- "A Statistical Ray of Hope for Coakley" -- from Pollster reader Harry Enten (aka commenter Poughies):

[Silver's] piece makes the point that the margin between Martha Coakley and Steve Brown could be overstating Brown's lead. Silver points out that polling in close (margins of 10 or less in the polls) Senate elections since 2000 in deeply blue states (as measured by the Cook Political Partisan Index) has by an average of 3.4 points underestimated Democratic candidates' margin of victories. In deeply red states, on the other hand, polling has underestimated Republican margins by 1.9 points.

Though he used a Pollster.com average in 2008 and Real Clear Politics average in 2006 and 2004, Silver used a simple average of all non-partisan polls conducted in the final two weeks compiled by Pollingreport.com in 2002 and 2000. Perhaps, he did not know, but a Real Clear Politics average is also available for 2002 and 2000.

I was interested what, if any, effect substituting the Real Clear Politics averages in 2002 and 2000 would have on Silver's results. Therefore, I decided to create a new dataset modeled after Silver's, but using the Pollster.com data in 2008 and Real Clear Politics average* in 2006, 2004, 2002, and 2000.

Using these new rules, the underestimation of Democratic margins in blue states stays the same at 3.4 points. This result is not surprising, as the blue states part of the dataset is small, and only three results are available from 2000-2002. The underestimation of Republican margins in red states drops from 1.9 to 0.9 points.

The overall average of underestimation drops from 2.3 to 1.5 points. In only 7 of 23 contests was the underestimation above 3 points. In no contest did the polling average incorrectly predict the winner due to underestimation of Democratic candidates in blue states or Republican candidates in red states. The only incorrect winner chosen was in the South Dakota (a red state) 2002 race when the polls predicted a victory by Republican John Thune.

The bottom line is that perhaps the polls are overestimating Brown's margin in Massachusetts. The limited data involving closely polled elections in blue states suggest that Coakley might do better than the polls suggest, but when you look at the larger dataset of red states, Coakley should not expect a bump.

*In some cases (such as Alaska 2004), no Real Clear Politics "average" existed. I just averaged all the polls listed on Real Clear Politics (and in the case of Louisiana 2002) conducted in the final two weeks. I, unlike, Silver use internal polls in these cases... modeling myself after Pollster.com's inclusion of them.

P.S. The Real Clear Politics web pages can be hard to find [but are available at these links] for 2002 and 2000.

He also passed along a table (in Excel and PDF formats) with the numbers.

It's also worth taking a closer look at the six races -- there were only six -- that are the basis for the conclusion that polls understate support for Democrats:

2010-01-18_538Table.jpg

Most interesting are the three races (Maryland and Rhode Island in 2006 and New York in 2000) that produced the biggest errors. Of these, the Rhode Island example is partly the result of an RCP average based on just two surveys conducted over the final weekend of the campaign. We used a simple last-5-poll average for Rhode Island that year, which showed Whitehouse winning by six, just one point off the actual margin. Just swapping the Pollster and RCP averages for that one race would cut the average variance to +2.5. Whatever the challenges of polling in Massachusetts, greater random error due to a shortage of final polls is not one of them.

That leaves two big errors affecting Ben Cardin in Maryland in 2006 and Hillary Clinton in New York in 2000. What's interesting about the errors in both contests was that the final round of polls had the percentage for the Republican candidate about right (within a point), but understated the support for the Democrat by about five percentage points. Even if we assume that a similar pattern will apply in Massachusetts tomorrow, the problem is that the six of the last seven surveys estimate Brown's support at 51% or greater.

 

Comments
Vicente Duque:


Very Interesting and Informative. To answer your question : I think Republicans are more interested in showing flags and strength. Democrats may be introverted, shy, timid and bashful.

Let's assume that Brown wins, Coakley loses.
But Obama wins, because he showed strength, youth, strong oratory, cleverness, compromise, obligation, stamina, vigor, he showed that he is still alive.

Rumors of Obama ( Political ) Death are greatly exaggerated !

The behaviour that Obama avoided was : Lazyness, Indifference, Negligence, Indolence, Passivity, Egotism, Egoism, Narcissism, Egolatry, Weakness, lethargic inactivity, slothfulness, slowness.

Obama is a winner, Obama wins.

And I am a Great Critic of Obama's Foreign Policy, but I am not so mean or debassed as not to acknowledge Obama's Greatness, ...... Yes, Obama is a Great President, and I pray for him, and that he corrects his course in certain matters.

But they guy is certainly great and good for America. I take off my hat before Obama !

Youth, Minorities, Politics :

Milenials.com

Vicente Duque

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR