October 15, 2006 - October 21, 2006
I have spent a lot time over the last few days pouring over the U.S. House polling numbers on Pollster as we have been at work on some sort of summary scorecard. But before we plunge back into the micro, district-level analysis, I thought it would be useful to do a bit of a review of the way things look at the national level. We do seem to be facing a surge of voter discontent with the Bush administration and the Republican Congress, but forecasting how that surge will impact the race to control the House is something of a puzzle.
Last week, I spoke at a forum sponsored by Campaigns and Elections magazine and was fortunate to share the stage with Democratic pollster Mark Mellman. He summed up this puzzle with a metaphor:
There's a big anti-Republican wave out there. But that wave will crash up against a very stable political structure, so we won't be sure of the exact scope of Democratic gains until election night. We really don't yet know which is ultimately more important -- the size of the wave or the stability of the structure.
So let's take a closer look at what we know now about the size of the coming wave and the stability of the structure.
Generic Ballot. National pollsters measure Congressional vote preference with a question commonly known as the "generic ballot," usually some variation of the following: "If the election for U.S. House of Representatives were held today, would you vote for the Democratic or Republican candidate in your congressional district?"
The goal of the question is to measure the total national vote cast for Congress. The good news for Democrats is that they have been leading on the generic ballot by margins of 10% or more since August, with a recent bump up in the aftermath of the Foley page scandal. However, the Foley bump is just icing on the Democratic cake. When my colleague Charles Franklin compared the trend in the generic ballot to past elections back in August (see especially the powerhouse graphic below), he noted that Democrats have not had a lead that "approached 10%, let alone exceeded it" since 1994.
That lead is important because historically, as Franklin showed in a subsequent post, the better a party does on the generic ballot the higher its share of the national Congressional vote. Unfortunately, however, that question proves to be a very blunt instrument in predicting seat changes, showing a huge historic variation of 45 seats on Franklin's chart. And as he points out, since 1946, the generic vote favored the Democrats just before 7 of the 8 elections that installed Republican majorities in the U.S. House.
There are two reasons for this imprecision. The first is that the generic question omits the names of the candidates, so the answers probably tell us more about general attitudes toward the two parties than about actual vote preference. Surveys I have seen recently that asked both the generic and actual votes produced a mismatch at the individual level of roughly 25%.
The second problem: Forget the survey question, for the moment. The national vote for Congress - the thing we are trying to predict -- is itself an imperfect predictor of seat changes. Franklin has the details (and as usual, the killer chart), but the following table shows some of the noise. Notice, for example, that the Democratic share of the Congressional vote was lower in 1998 than 1996, but they managed to pick up five seats. Similarly, Democrats improved their total vote margins between 2002 and 2004, but still managed to lose a net three seats.
Franklin also provides evidence that number of seats won by each party has "been MUCH less responsive to changes in votes since 1994 than in the previous 46 years . . . I would be very reluctant to assume that the historic relationship between votes and seats is still true."
The weak relationship between the vote and the seat count beaks down because of the "stability in the system," that Mark Mellman talked about. We'll get to that.
Given all this imprecision, why pay attention to the national generic ballot at all? Because comparable national surveys are conducted more often, with larger sample sizes and more rigorous methodologies than many of the surveys we are seeing at the district level. If a last minute change occurs in the national political environment ("the size of the wave") the national surveys will show it first.
[Editor's note to self: This is a blog, not a book. so I'll stop here and pick up with a look at the "enthusiasm gap" in the next post].
Yesterday, our Slate update shifted to a district-by-district focus on races for the U.S. House. Our initial tally shows Democrats right on the edge of statistically meaningful leads in enough districts to take control of the House, and Democratic challengers running within the margin of error in many more. While we are working to creating a more comprehensive scorecard, I want to say a bit more about how our initial count given the limitations of the data.
For now we are focusing on non-partisan surveys conducted since late August (which allows inclusion of the first wave of Majority Watch surveys conducted August 27-29). We have excluded the surveys sponsored by campaigns or the party committees (including those conducted by my firm).
Looking at the survey averages in districts with two or more polls available, we see Democrats leading beyond the margin of error in ten districts currently held by Republicans (the number of surveys analyzed is included in parentheses):
In addition, we see statistically significant Democratic leads in four more districts held by Republicans surveyed only once by non-partisans since the summer (all four were polled by the Majority Watch project):
[Note: We inadvertently omitted Ohio-18 and North Carolina-08 from last night's initial Slate update].
Perhaps more troubling for Republicans is that we see no Republican leading in any district currently held by a Democrat. Moreover, of the 23 Republican held seats currently rated as "toss-ups" by the Cook Political Report, Democrats lead by significant margins in 9, Republicans leading in none
just one (Minnesota-02).** The remaining 13 Republican "toss-up" seats look too close to call based on available data. And that says nothing of the 31 Republican seats that Cook rates at "lean" or "likely" Republican, where public polling is scarcer still.
Of course, readers should remember the limitations of these data. Any one poll can produce an odd or contradictory result and many of the polls conducted in September may already be stale. Consider, for example, Iowa's 1st District (an open seat currently held by Republican Jim Nussle). A Majority Watch poll in late August showed the Democratic candidate Bruce Braley leading Republican Mike Whalen by thirteen points (54% to 41%). Two weeks later, a DeMoines Register/Selzer poll conducted two weeks later had Braley ahead by seven (44% to 37%). Then a Zogby poll at the end of September showed Braley trailing by thirteen (34% to 47%). And finally, a poll conducted by my firm last week for the Democratic Congressional Campaign Committee had Braley ahead by 10 (48% to 37%). So over the course of six weeks, polls in one district show everything from a 13 point Democratic lead to a 13 point deficit with undecided percentages ranging from 5% to 19%. These differences are far beyond sampling error and almost certainly the result of differences in pollster methodology. And that's just one district.
Again, we are working on creating a House summary scorecard for Pollster.com, but it will be limited by both the relatively small number of public polls and their inevitable conflicts.
**Correction: I wrongly included MN-02 (rather than MN-06) among the list of seats rated a "toss-up" by the Cook Political Report. Four polls in MN-06 since mid-September give Democrat-Farm-Labor candidate Patty Wetterling an average lead of one percentage point (45.7% to 44.7%) over Republican Michelle Bachman. Apologies for the error.
No time to write this up. Come back for that later. Bottom line: CNN at 36%, Gallup and Zogby at 37%. Trend estimate falls to 35.8% This is a stunning decline that doesn't yet seem to have reached bottom. Can the White House do anything to stop it?
Note: This entry is cross-posted at Political Arithmetik.
With the addition of House race data to Pollster.com, it is a good time to talk about the difficulty of measuring the status of the race to control Congress at the district level. Political polling is always subject to a lot of variation and error (and not all of it the random kind), but Congressional district polls have their own unique challenges.
First, we are tracking something different in terms of voter attitudes an preferences than in other races, particular contests for President. Two years ago, voters received information about George Bush and John Kerry from nearly every media source for most of the year. Huge numbers of voters tuned in to watch live coverage of nationally televised candidate debates. In races for the Senate and House, news coverage is far less prevalent and voters pay considerably less attention until the very end of the campaign. Even then, voters still get much of their information about House candidates from paid television and direct mail advertising.
Of course, in the top 25 or 30 House races, the candidates (and political parties) have already been airing television advertising. However, if you expand the list to the next 30-40 races that could be in play, the flow of information to voters drops off considerably. Middle-tier campaigns in districts in expensive media markets (like New York or Chicago) will depend on direct mail rather than television to reach voters.
So generally speaking, voter preferences in down ballot races are more tentative and uncertain. The (Democratic affiliated) Democracy Corps survey of Republican swing districts released last week reported 26% of likely voters saying there is at least a "small chance" they may still change their minds about their choice for Congress. When they asked the same question about the presidential race in mid-October 2004, only 14% said they saw a "small chance" or better of changing their mind about voting for Kerry or Bush.
This greater uncertainty means that minor differences in methodology can have a big impact on the results. Specifically, pollsters may vary widely in terms of the size of the undecided they report depending on how hard they push uncertain voters.
Second, the mechanics of House races polling can be very different from statewide methodology. The biggest challenge involves how to limit the sample to voters within a particular House district. In statewide races the selection is easy. Since area code boundaries do not cross state lines, it is easy to sample within individual states. So most of the statewide polls we have been tracking use a random digit dial (RDD) methodology that can theoretically reach every voters with a working land line telephone.
No such luck with Congressional districts, whose gerrymandered borders frequently divide counties, cities, even small towns and suburbs. Since very few voters know their district numbers, pollsters use a variety of strategies to sample House districts. Most of the partisan pollsters, as well as the Majority Watch tracking project, use samples drawn from lists of registered voters (sometimes referred to as "registration based sampling" or RBS). These lists make it easy to select voters within a given district, but the lists frequently omit telephone numbers for large numbers of voters (typically 20% to 40%
30% to 50%**). Remember the real fear that RDD surveys are missing cell-phone-only households? Right now the missing cell phone households represent roughly 6-8% of all voters. Lists, obviously, miss many more. If the uncovered households differ systematically from those with working numbers on the lists, a bias will result.
Again, most partisan pollsters (including my firm) are comfortable sampling from lists, because the benefits of sampling actual voters within each District appear to outweigh the risks of coverage bias (see the research posted by list vendor Voter Contact Services of a sampling of arguments in favor of RBS). Media pollsters are generally more wary. SurveyUSA, for example, conducted a parallel test of RDD and RBS in a 2005 experiment that found a large and consistent a bias in RBS sampling that favoring one candidate. "SurveyUSA rejects RBS as a substitute for RDD," their report read, "because of the potential for an unpredictable coverage bias." So in House polls they often use RDD and screen for voters in the given district based on voters' ability to select their incumbent member of Congress from a list of all members of Congress from their area.
These various challenges have made many media outlets and public pollsters wary of surveys in House races. As of two week ago, we had logged more than 1,000 statewide polls for Senate or Governor into our Pollster.com database for 2006. As of yesterday, we had tracked only 173 polls conducted in the most competitive House races, but as the table below shows, only 47 of those came from independent media pollsters using conventional telephone methods
Nearly half of all the House race polls come from two automated pollsters: SurveyUSA (23) and especially the Majority Watch project of RT-Strategies and Constituent Dynamics (56). Also, more than a quarter of the total (52) are partisan surveys conducted by the campaigns, the party committees or their allies, with far more coming from Democrats (44) than Republicans (8).
The sample sizes for House race surveys are also typically smaller. While national surveys typically involve 800 to 1000 likely voters, and statewide surveys 500 to 600, many of the House polls involve only 400 to 500 interviews (although the Majority Watch surveys have been interviewing at least 1000 voters in each district).
Finally, very few districts have been surveyed by public pollsters more than a few times since Labor Day. Only two of the 25 seats now held by Republicans rated as "toss-ups" by the Cook Political report have been polled 5 or more times. Most of these critical seats have been polled 2 to 4 times. Put this all together, and the results are likely to be more varied and more subject to all sorts of error than other kind of political polls. After the 2004 election, SurveyUSA put together a collection of results for every pre-election public opinion poll released in the U.S. from October 1 to November 2, 2004. Their spreadsheets included 64 House race surveys, and their calculations of the error of each survey indicate that those few House races had more than double the error on the margin (5.82) than the polls conducted in the presidential race (3.43).
All of which goes to say that while we too will be watching the House polls more closely over the next three weeks, for all the tables and numbers, we know far more about these races than meets the eye. More on what we do know tomorrow.
**Correction: Colleagues have emailed to point out that quoted match rates for list samples have improved in recent years and now typically range from 60% to 80%. I won't quarrel, although I have had past experiences where quoted rate exaggerated the actual match once non-working numbers are purged from the sample.
[Editor's Note: Today's Guest Pollster Corner item comes from Anna
Greenberg, Vice President of the Democratic polling firm Greenberg Quinlan
Along with taxes and security, immigration represents one of the pillars of Republican attacks on Democratic attacks this year. Regardless of the district or state, an inflammatory attack on Democrats for allegedly supporting amnesty and Social Security benefits for illegal immigrants fill the airways and mailboxes. But the data are not so clear that attacking Democrats on immigration is terribly effective, at least in many races. While many Americans hold hostile views towards immigrants and are deeply disturbed by the notion that illegal immigrations receive government benefits (which is not entirely true), the debate as it has played out has not necessarily benefited Republicans.
In a recent national survey, we found that Democrats actually have an advantage on the immigration issue among all voters, and Independents, in particular. Moreover, in an exercise where we test two statements -- one that presents the Republican attack on Democrats for supporting amnesty and benefits for illegal immigrants and one that presents the Democratic argument that is supportive of border security and points out the federal failure on immigration -- the Democrat wins the argument by healthy margin.
The Republican failure to effectively exploit anti-immigrant views stems from the fact that the issue has more nuance than "amnesty versus no amnesty." While many voters believe illegal immigration is a real problem, they have real differences about the right solution. For example, no matter how many activists support the "security fence" along the border, voters have real questions about its effectiveness and cost. Many also agree that simply deporting 11 million illegal immigrants is not practical and some believe it would be inhumane. Republicans have also been divided on the issue and are not speaking with a clear voice; most national polls going into the mid-term elections showed that Bush and the Republican had no clear political advantage on the immigration issue, in part because of the conflict between Bush, McCain and the House.
Finally, some Republicans are challenged by just sounding too extreme (not to mention alienating Hispanic voters). I would point to Arizona's 8th district as an example where extreme views on immigration have hurt the Republican candidate (full disclosure: I work for the Democratic candidate, former state senator Gabrielle Giffords). But even in other races where the issue is fully engaged by less extreme candidates on immigration -- the PA Senate race, IL 6th, and IN 2nd are some examples -- Democrats have certainly weathered the attacks or even increased their leads after being attacked on immigration. At the end of the day, other issues might simply be crowding out immigration in these competitive races, but I would argue that Republicans have miscalculated the power of the issue and that Democrats have strong arguments that counter their attacks.
A staple of modern American politics is the article or column that inevitably follows any landslide election speculating whether this particular victory heralds a national realignment that will reshape politics for decades. It rarely does. Make what you will of the fact that this year's version, by David Kirkpatrick in today's New York Times, appears three weeks before the election, but whatever you think of the timing of this piece, do not overlook the remarkable, must-click graphic that accompanies it.
The chart plots data on the current party identification of Americans compiled from more than 23,000 Americans surveyed by the Pew Research Center during 2006. With that many cases, the Pew pollsters were able to tabulate a result for party identification (including independent "leaners") for each birth year and plot the results. What results is a remarkable picture of the politics in play as each age group emerged into adulthood. The most Republican cohorts are those who came of age during the administrations of popular Republicans: Dwight Eisenhower, Ronald Reagan and George H.W. Bush. In contrast, the current crop of 18-24 year olds, according to the Pew Research center data, is the most Democratic leaning group in the population.
The chart provides graphical evidence of the slow rolling realignment that is always at work as new young voters gradually replace their elders. Political scientists generally agree that young people tend to acquire political beliefs, including their partisan attachments, in their 20s. As Kirkpatrick writes, "voters typically develop a party preference based on the political atmosphere at the time they come of age and grow more attached to that party over the course of their lives." Once acquired, a true sense of party changes rarely changes, although some voters are less attached to political parties than others (as I speculated on Friday, some will shift back and forth on surveys depending on the politics of the moment or the wording or context of the survey question).
The gradual shifts toward Democrats since 2005, and the small recent spike I wrote about Friday, are almost certainly not harbingers of some coming realignment. They will likely be as fleeting as the shifts toward the Republicans just prior to the 2004 elections. The bigger question is what these changes portend about the still outcome of the elections next month. The implications for Republicans certainy seem dire, but we will not know for certain until November 8.