Articles and Analysis


Gallup vs. Newsweek on the Generic

Topics: Gallup , Generic House Vote , Newsweek

Two national polls released today and over the weekend report very different results leading to very different conclusions:

On Friday, under the headline "Democrats May Not Be Headed for Midterm Bloodbath," Newsweek reported results from a new national poll of registered voters showing Americans evenly split (45% to 45%) on the question of whether they would vote for the Democratic or Republican candidate for Congress in their district.

This afternoon, Gallup released another national survey of registered voters, also conducted last week, showing Republicans with an "unprecedented 10-point lead" (51% to 41%), the largest Republican advantage Gallup has measured in its nearly sixty years of tracking the so-called "generic ballot."

So what's going on?

Much of the gaping difference between the two polls is probably explained by the usual random variation that affects all polls. Use your mouse to poke around our interactive chart (posted below), and you will soon discover that the latest Gallup survey result is more favorable for the Republicans than most, the Newsweek poll is similarly more favorable for the Democrats and that both fall within the typical range of variation, amounting to +/- three or four points from the trend line. Our overall trend estimate based on all of the available polls gives Republicans a 5.2 percentage point advantage (46.8% to 41.6%)

We could obsess further over the consistent differences ("house effects") among pollsters, but what is far more important, is that the averages show a GOP lead that has been trending in the Republican direction all summer. That trend is consistent with the historical pattern identified here on Friday by political scientists Joe Bafumi, Bob Erikson and Chris Wlezien, the "electorate's tendency in past midterm cycles to gravitate further toward the "out" party over the election year."

Moreover, you see the same trend even if we drop all Newsweek and Gallup polls, plus all of the Internet-based surveys and automated surveys (including Rasmussen), and focus only on the remaining live-interviewer telephone surveys, as in the chart below. The margin for the Republicans is virtually identical (46.6% to 41.4%).


So while the "unprecedented 10-point lead" reported by Gallup probably exaggerates the Republican lead, any result showing a net Republican advantage on the so-called generic ballot is bad news for Democrats. Bafumi and his colleagues estimated their 50-seat gain for the Republicans assuming a two-point advantage for Republicans on the generic ballot, which they project will widen to a six-point lead by November. If the Republican lead on the generic ballot is already that wide (or close), their projection for the Democrats would worsen.

[Cross-posted to the Huffington Post].



Come on. Are you honestly placing a Newsweek poll on the same level as a poll from the most trusted news organization in the US? Newsweek has quite possibly the single worst track record there is. Furthermore, an extremely compelling case has been made that the numbers in the Newsweek poll are completely bogus:


Given the sheer number of polls that have come out recently showing Republicans with a lead on the generic ballot, it is quite obvious that a poll showing a tie in the generic ballot is pure garbage. And the link that I provide above shows pretty conclusively that the poll results are fraudulent.


Where are the regional polls? All of this national generic polling nonsense is, in my opinion, useless this time around as any kind of a predictor, since we know there are enormous variables in regional political preferences.

I am aware of only one poll, about a month ago, that showed generic ballot choices on a regional basis, and it showed Republicans winning by some 30 points in the south, but losing in all 3 other sectors of the country. That sounds very logical to me, particularly when looking at some specific races, which aren't necessarily showing any kind of a "tidal wave" election, but merely a pretty typical off-year loss for the president's party.



"particularly when looking at some specific races,"

What individual house polling that exists seems to refute the wave narrative.

Ie: SD-AL. The democrat is down by 9 according to Rasmussen, the only firm to poll the race. But listening to the narrative you'd think she'd be down by 20. Considering she only won the contested elections she was in by 6 or 7, down by 9 is not insurmountable. She will probably be one of the losers but it's not a foregone conclusion yet.


Henry Henwy:

I sincerely hope that MSNBC either responds to the concerns cited or that there is further investigation of their polling results if they are obstinate about it. The issues raised by Dr Eberly are pretty troubling and make a good prima facie case for something fishy. I would really like to know how they reconcile the results they reported with the crosstabs they themselves provided.


Henry Henwy:

Duh. I meant Newsweek, not MSNBC.



While the newsweek poll is likely to be too positive for the democrats, Professor Eberly's analysis ignores a couple of fairly obvious points, firstly if you weigh for "gender, age, education, race, region, and population density", the party id of the weighted sample will most likely change. (Indeed it does from 33D 33R 29I to 36D 32R 27I). This is not something 'fishy' it is simply due to a lower response normally by minorities and the young who tend to be more likely to be democrats.

Secondly the party-id numbers do not add up to 100%. There is 5% of the RV sample who are non-R/D/I. Before these 5% are counted the ballot question would come out 44.39%R- 42.59%D. So the 5% could easily make it up to 45% a piece especially when you include rounding.

"Newsweek has quite possibly the single worst track record there is"

Come now what about Zogby Internet, or ARG? Newsweek seems to be fairly average, at least according to 538 ratings, although the sample number is small at 16.


Matt Sheldon:

This post illustrates the problem with the direction of Pollster.com.

The unweighted Newsweek poll shows a 6 point GOP lead.

Comparing a dishonestly weighted Newsweek poll with an unweighted Gallup poll is not a reasonable comparison.

Weighting polls gives the pollster the right to dictate the outcome, and Newsweek is not really a mainstream publication anymore. Jonathan Alter is a complete joke of a pundit.

This sort of "inside baseball" commentary should be the purview of Pollster.com. You certainly won't get it from Nate Silver, who is highly selective in his pollster criticisms. His rankings are not to be trusted.

With regards to regional polling, which is the common retreat of the left these days, there has been ample refutation if this point.

There was an in depth poll of contested districts which skewed toward Democrat-held seats across geographies. They were losing. Substantially.

We also have the fact that the GOP won contested races in MA and NJ by comfortable margins, and the electoral situation for the Democrats has only deteriorated since.

This election is headed toward a wipeout for the Donkeys, there is no longer much doubt of that.



"Comparing a dishonestly weighted Newsweek poll with an unweighted Gallup poll is not a reasonable comparison.

Weighting polls gives the pollster the right to dictate the outcome, and Newsweek is not really a mainstream publication anymore."

I don't think newsweek is weighted by party-id. All polls will be weighted by such things as age, gender, race etc, including gallup. Both polls seem to be about the same amount off the average which shows about a 5% lead for republican at the moment so it would seem a reasonable comparison to make, but as Mark points out the trend is what is important.

The trend certainly suggests you are probably right, and that the november election are increasingly looking like a very harsh enviroment for the democrats.


Field Marshal:

Duh. I meant Newsweek, not MSNBC.

Not much of a difference. Both are worth about a $1.


Matt Sheldon:


The Gallup poll is unweighted. The Newsweek poll is weighted.

In the unweighted results, the GOP has parity in Party ID.

This disappears once the weights are applied and the Democrats have a substantial advantage.

Newsweek can weight their survey however they see fit, that is their call.

However, when comparing two polls there should be dome discussion of the differing methodologies. In this case, it appears to be the critical factor.

An expert polling site should be able to see right through this to help explain the variance.

Pollster.com missed it entirely. One wonders why.

Those who will not look will never see.


Jennifer MacAdam:

I'm web-polled by Gallup all the time. They always ask me if I approve or disapprove of whoever the president is at the time. I didn't like Bush and I do like Obama, and I NEVER am polled about politics. Who says that Gallup is credible?


Mark Blumenthal:


You're wrong. At the bottom of that and every Gallup release you'll find this disclosure:

Samples are weighted by gender, age, race, Hispanic ethnicity, education, region, adults in the household, cell phone-only status, cell phone-mostly status, and phone lines. Demographic weighting targets are based on the March 2009 Current Population Survey figures for the aged 18 and older non-institutionalized population living in U.S. telephone households.

Neither Gallup nor Newsweek weights by party. If that confuses you, read the post again.


Post a comment

Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.