Pollster.com

Articles and Analysis

 

Democratic Surge? Part II

Topics: 2010 , Gallup , Generic House Vote

Last week, I argued that a reported “jump” for Democrats in Gallup’s weekly tracking of the national generic U.S. House ballot was most likely a statistical blip. I boldly predicted that “more data” this week would “likely settle the issue.” That latter assertion turned out to be wrong, as the issue isn’t settled, but I’m still not convinced that we’re seeing a real shift in voter preferences nationally.

Let’s review: Generally speaking, the generic House ballot is a poll question that asks registered or likely voters whether they would support "the Democratic Party's candidate or the Republican Party's candidate" in their congressional district if the election were held today. Since March, Gallup has released a weekly result based on roughly 1,600 interviews of registered voters that has averaged a 46% to 46% dead heat, but mostly varied within the expected margin-of-error range of plus or minus 3%.

Last week’s Gallup result showed Democrats with a six-point lead (49% to 43%), a result that I argued was likely the sort of random statistical “blip” we should expect from time to time with this sort of tracking survey. This week, Gallup reported a Democratic margin is a slightly narrower four points (48% to 44%), but Gallup’s analysis noted that it “marks the second straight week in which Democrats have held an edge of at least four percentage points” and “the first time either party has held an advantage of that size for two consecutive weeks” in Gallup’s tracking.

So this week’s data doesn’t resolve things. As Charlie Cook writes today, these results mark “one of those periods of uncertainty” where those of us who watch polls closely are unsure whether the results “signal a key turning point in public opinion…just a hiccup, a passing blip…[or] an outlier poll, a statistical anomaly that is the political equivalent of a false positive medical test.”

I’m still dubious that the we are seeing a real change in voter preferences. First, while this week’s Gallup’s numbers do tend to confirm last week’s upward turn, they are also statistically consistent with the 46%-to-46% result that Gallup has shown on average since March. Combining samples for the last two weeks might yield a statistically significant difference from the average, although it would be close.

Second, none of the other pollsters that fielded national media surveys in recent weeks confirm a “jump” in the Democratic direction since the passage of financial reform legislation on July 16. While these surveys have “house effects” that produce different results over time, we can look at trends for individual pollsters. Six (Ipsos, Rasmussen, CNN/ORC and Zogby) show nominal shifts in the Republican direction as compared to their average result earlier this year, and one (YouGov/Polimetrix) shows no change. Only Gallup shows a shift to the Democrats.

Third, Charlie Cook took the next logical step and informally “canvassed several pollsters who see large quantities of data from around the country.” These are the campaign consultants that have been doing benchmark and tracking surveys for their clients over the summer. “None,” he writes, “seems to have detected any shifts in the past two weeks.”

Charlie also makes the point that he sees “no defining event has taken place,” including passage of the banking bill, “that would have triggered a significant shift in this year’s race” and says the pollsters he talked to are also “at a loss in figuring out what would have triggered a change.” Count me as similarly puzzled.

That said, I am not arguing that we ignore the Gallup data. Aside from its well-deserved reputation, Gallup is also the only organization polling on the generic ballot in recent weeks which interviews Americans on their landline and cellphones. It would be surprising for the cellphone interviewing to make that big a difference, but we can’t rule that possibility out.

So, as my friend Charlie Cook counsels, we need to “sit tight” and wait for more data.

[Cross-posted at Huffington Post]

 

Comments
StatyPolly:

Perhaps the cellphone issue does make a difference, but as a conservative partisan, I've been more than happy to see what I consider more than a "nominal shift" in past two weeks of generic ballot polling.

Pollster.com generic graph is at all time high (for this cycle) in favor of GOP at 44.7/41.9.

/polls/us/10-us-house-genballot.html

Taking out the outlier that Gallup has been the past two weeks, the gap extends further to 44.9/40.6.

Additionally, Mark probably wrote this piece before Fox's latest generic ballot came out, that had GOP at an astonishing for RV's +11. Also, Quinnipiac poll that was conducted at roughly the same time as Zogby and YouGov matched GOP's best showing this cycle at +5.

To my naked eye, it only adds to the narrative that Gallup has been out in left field for two straight weeks, since six of eight remaining pollsters showed from nominal to significant movement towards GOP.

Of course, the even bigger picture is that even if Gallup did get it right, and all the rest got it wrong, it still points to GOP lead for control of the House, once Gallup's own recommended likely voter adjustment is applied to their registered voter polling.

____________________

kariq:

I hate to throw cold water on the celebration of the Fox poll results and how they improve the picture for Republicans, but the Fox poll should really be thrown out. Not because Fox is a "lean conservative" poll, but because their polls are notoriously bad. In 2004, for example, their final presidential poll showed John Kerry leading George W. Bush; I believe they were the only ones to show a Kerry lead, and we all know how that worked out.

The recent poll showing the GOP ahead in the generic ballot by 11 should be thrown out of any average. They polled more Republicans than Democrats, and no one, not even Rasmussen, shows a higher Republican identification than Democratic one. Clearly, it's a bad poll and I think it should be eliminated from the average.

If you take Fox out, you get GOP 44% and Dems 43%, a statistical tie and basically unchanged from where it's been for 5 months or so.

____________________

CompCon:

That's true, kariq, but if you want to play those games, you can take out just the YouGov internet poll (which has always been horrendously bad) and the numbers change to GOP 45% and Dems 40%. So yes - one poll can skew the results - and some of the do.

Also, if you take out both Fox and YouGov, you get exactly the same results as you get with all polls included.

So the bottom line is that you leave all of the polls in there and you get a good picture of what the real trend is.

____________________

AlanSnipes:

All of these conflicting polls and biased interpretation by partisans on both sides means nothing.
I believe that these polls will stay like this and we will see what happens on election day.

____________________

kariq:

You can certainly make a case for eliminating all internet polls, though I would say YouGov is not the worst of offenders in that sense.

____________________

kariq:

Sorry for the second post in a row, but I did want to point out that your comment about YouGov's record isn't really accurate. When Nate Silver reviewed their performance, he found YouGov's internet polling was comparable to phone polls in accuracy. So the argument for removing them from the average while still including Zogby's internet polling is weak.

If your skeptical of internet polls, remove them all, certainly. But it makes no sense to remove YouGov, who has a fairly good record, while keeping Zogby, who has a piss-poor record.

____________________

John:

"you can take out just the YouGov internet poll (which has always been horrendously bad)"

YouGov haven't been that bad, they are slightly above average on the 538 ratings both in the PIE score and the raw score. Zogby Internet on the other hand........

The party-id of the recent fox-news did seem somewhat skewed suggesting a bad/unlucky sample, however I suspect that if we could see the full breakdown of the recent +6D Gallup poll it might also have a somewhat skewed/unlucky sample.

____________________

CompCon:

Nate Silver didn't do any rating on generic ballot or healthcare reform - his were only on general elections. YouGov might do fairly well in those but they lean fairly heavy to the left on those other polls.

Besides that, Nate Silver's rating system is nothing more than figures manipulated with arbitrary factors to fit his gut feeling of which ones he likes and which he doesn't.

What I'm saying, kariq, is that you think the Fox poll should be removed because you think it is skewing the results, but the YouGov poll is skewing the results by exactly the same amount in the opposite direction. So you want the right-leaning one removed but leave the left-leaning one. Why is that? They both affect the composite equally.

____________________

kariq:

You misunderstand me. I would always remove Fox polls from every average, regardless of the results they gave, just as I would always remove Zogby internet polls from every average regardless of the results they give. That's because those polls have dramatically poor track records.

Health care shouldn't be part of a judgment on a pollster's accuracy since there is no election where that accuracy can be checked. You seem to assume they are wrong on health care because the results are more to the left than you want, but absent an election to verify the accuracy of the polls, there's no way to be certain if Rasmussen or YouGov has a better measure of the public opinion on these issues.

A house effect, regardless of its direction, wouldn't cause me to remove a poll from an average, only a record of higher than average bad polls. And both Fox and Zogby internet polls have very poor records.

____________________

John:

"Nate Silver didn't do any rating on generic ballot"

I didn't realise the YouGov had done previous polling on the generic ballot, do you have a link to the past polls?

"Besides that, Nate Silver's rating system is nothing more than figures manipulated with arbitrary factors to fit his gut feeling of which ones he likes and which he doesn't."

You may well disagree with Nate Silver's methodology, but it was not exactly arbitrary founded on nothing, given most of the 'factors' I assume you are talking about were based on a regression model. But even if you disagree with his system, (it can be pretty unfair to pollster with only a few polls who do not sign up to the NCPP/AAPOR disclosure, but then again it is meant to predict future accuracy rather than being fair), you can still compare the 'raw score' in which YouGov actually does relatively better in.

To be honest I am not sure about the merits of removing the fox poll either. Most pollsters have had there misses and while overall it's past performance has been slightly below average it is nothing compared to zogby interactive. If you did want to remove fox for being inaccurate in the past, then you probably should remove all other pollster with worse record than them such as Suffolk, Democracy Corps, Insider advantage, zogby telephone, Battleground and ARG. (Actually removing the last one may not be such a bad idea)

____________________

CompCon:

If you look at the composite page, /polls/us/10-us-house-genballot.html, you'll see links to YouGov's generic ballot polls. They do one every week. They always seem to lean toward the democrats compared to others run at similar times. Since it is a weekly poll, their numbers exert a lot of influence on the pollster.com composite picture.


@kariq: "higher than average bad polls."

Bad poll based on what? Because of the party affilitation? Who actually knows what the real number for that is? We know it isn't the 2008 model which has changed drastically since then - although a lot of pollsters still use that.

I agree that the fox poll looks like a fluke - like the gallup polls from the past 2 weeks. If you do some reading at gallup.com, they explain how their polls might have bigger random spikes because that is the nature of randomness. If you go back and look at previous Fox polls, you will see that they are very close to all the other polls during the same periods.

____________________

John:

@Compcon

But those polls are for this cycle and therefore we do not know if they are accurate or not. You stated that YouGov have always been 'horrendously bad', is that just based on this cycle generic ballot polls leaning (or having a house effect) to the democrats? Rasmussen has consistently had a larger house effect in the other direction, but until the election we will not know which is right or whether their polls will converge as the likely voter screen is applied or changed.

____________________

kariq:

Actually, the article where Silver's comments about the accuracy of YouGov polls was based on their performance in the last election cycle, which was about on a par with phone interviews. He didn't adjust the model at all, simply checked their polls against actual results. You can read it for yourself: http://www.fivethirtyeight.com/2010/06/zogby-broke-internet-but-it-can-be.html

CompCon: Who actually knows what the real number for that is?

I mentioned the party ID on this poll because it's easy to see how they screwed this one up: too many Republicans. But if you want to go there:

How do we know the actual numbers? Check the aggregate of all polls, throw out obvious outliers. Actually, do this now. It might show you what I'm talking about.

Fox lists the results only for registered voters, and their most recent result is Rep 41, Dem 37. They are quite literally the only pollster that shows more Republicans than Democrats. When every other pollster out there shows the same result, and that result is different from Fox's, who's most likely to be wrong?

To the best of my knowledge, Opinion Dynamics' accuracy hasn't been rated individually in 2010. I didn't see it on Silver's pollster ratings, though I may have missed it. I'm willing to be wrong about that. Lacking that, I can only go by my own impression, which is that I've seen more screwy polls out of Fox than most pollsters. Your mileage may vary.

Oh, and please do remove ARG from pollster averages. Always.

____________________

CompCon:

Ras and Gallup are out with their weekly generic. Rasmussen has only changed by 2 points since last week - republicans +8 using a likely voter model.

Gallup has swung back toward republicans just as wildly as it did toward democrats 2 weeks ago. It is now republicans +5 using a registered voter model. That is after a +4 for democrats last week and a +6 the week before.

This is gallup's biggest one week swing yet. A 9 point swing toward republicans in a single week.

Maybe this will end the talk of "the democrat surge" we've been reading about. Looks like Gallup had 2 fluke weeks in a row.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR