Pollster.com

August 24, 2008 - August 30, 2008

 

"Bounce" Update

Topics: Barack Obama , Bounce , Bump , Conventions , Gallup Daily , John McCain , Sarah Palin

I spent much of yesterday in taxis, airports and planes, and most of the last 24 hours sleeping and reintroducing myself to my family after a five day absence. So my apologies for being away when many of your are curious about the "bounce" and reactions to Republican VP choice Sarah Palin.

Let's start with "the bounce." The Gallup Daily tracking survey conducted Wednesday through Friday shows Barack Obama leading John McCain by eight points (49% to 41%) up from the 45-45% tie measured the full week before the convention. The Rasmussen Reports automated survey conducted over the same three nights shows Obama with a four point lead (49% to 45%), after they had him "leading by just one or two points for most of August."

The most important question, tossed my way by my colleague Marc Ambinder, is whether this shift represents a momentary "spike" or a real and persistent change in voter preferences. And the short but frustrating answer is, there is simply no way to know for certain right now.

One problem is that we have a very limited sample of past convention "bounces" to examine -- a sample size of 10 elections and 20 conventions since 1968 -- and the patterns within that sample have been inconsistent. Some bounces persist, some fade almost immediately.

Another problem, which I explored in a column earlier in the week, is that almost all of the "bounces" of conventions past were measured by multi-day polls conducted a week after the convention without another convention underway. This time, the virtual overlap of the two conventions prevents us from obtaining post DNC "bounce" numbers that are comparable to past measurements. So my recommendation is to avoid historical comparisons.

Finally, consider a few reasons why conventions bounces are sometimes just momentary blips.

First, most pollsters will tell you that attitudes are likely "in flux" at times like these. Watching one side of an argument for an hour or more might leave an uncertain voter "leaning" in one direction for a few days only to shift back in the other direction after hearing from the other side a few days or weeks later.

Second, the kinds of people likely to have been at home the last few days may have been a bit different than those who were away (and thus not part of the survey). Partisan Democrats and Obama supporters may have made it a point to be at home n order to watch the convention. Also, those who just happened to be home (for whatever reason) were probably more likely to have watched the convention and the news generally than those more likely to have been away (for whatever reason). The combination may result in a momentary "response bias" favor ingof the Democrats.

It is not surprising that the Rasmussen survey shows a smaller shift, since they weight by party identification. If the "bounce" is solely the result of a non-response bias toward Democratic identifiers, party weighting in this situation would make sense. But we don't know that it is. Conventions bounces that are real and persistent often shift party identification and candidate preference. In September 2004, for example, five national surveys conducted entirely after the Republican convention showed increases in Republican identification. Pollsters that weighted by the pre-convention party ID averages would have artificially suppressed the size of Bush's bounce.

So, like it or not, we really won't have a sense of what these shifts mean -- and what they portend for the rest of the campaign -- until the Republican convention ends and the dust starts to settle in about 10 days.

As for the polls out today testing reactions to Sarah Palin, I hope to post more tomorrow, but for now suffice it to say that I put very little stock in one-night quickie polls conducted on the Friday night before a three-day, holiday weekend. The one from Gallup is probably the best of the lot, if only because they provide comparable results from prior one-night quickie polls conducted to evaluate vice-presidential nominees (although most of these were fielded on week-nights). But take it all with huge grains of salt. With someone as unknown nationally as Sarah Palin was just 48 hours ago, first impressions can be exceptionally fleeting.


Gallup Daily - The Worst Thing in 10 Years?


In Mark Blumenthal's post on how David Plouffe is polling for Barack Obama, the Democratic nominee's communications director Dan Pfeiffer is quoted as saying that "the Gallup Daily is the worst thing that's happened to journalism in 10 years." Gallup's Frank Newport predictably rejected the comments, claiming that Pfeiffer's comments "are the same types of sentiments that have been expressed since George Gallup's first presidential polls in 1936."

 

I don't think Frank is correct in his boiler plate response. It is not useful to dismiss all criticisms of polls these days as the same old tired comments of seven decades ago that have long been discredited. If I understand Mark's blog correctly, Pfeiffer and Plouffe object to the Gallup Daily because it does not, contrary to Frank's assertion, provide an accurate description of where the presidential race stands today.

 

According to Mark's post, Plouffe claims that the topline polling data aren't especially useful (they "don't tell you anything"). Instead, the campaign focuses on who are the "true undecideds," and what messages will persuade them to vote for Obama. Knowing how many undecided voters there are is an integral part of understanding the presidential race. That's true for the campaigns, and it is no less true for political observers and the public.  

 

But Gallup refuses to measure the undecided vote, and instead gives a hypothetical description of a presidential race, "if the election were held today" - showing us that 95 percent of voters have already made up their minds. But the election is not being held today, and the Gallup Daily does not tell us the truth about how many voters are - at this point in the campaign - committed to a candidate, and how many voters have yet to make up their minds. From Plouffe's and Pfeiffer's point of view, the Gallup Daily is useless - even in understanding the national sentiment.

 

Frank claims that the public needs "independent polling" so that it doesn't have to rely on "campaign operatives' self-promoting insights on where the race stands." I  couldn't agree with him more. But the public needs accurate independent polling, which gives the public a full picture of where the presidential race stands. Gallup Daily does not do that. But it could.


State Battlegrounds and Home Grounds




























A quickie from Detroit Metro Airport.

Mark Blumenthal reported on an interview with Obama campaign manager David Plouffe yesterday at Pollster. Plouffe discussed the 18 states the Obama campaign sees as their target states, and Mark reported what states those were in his post.

Here we take a quick look at the polling in those states. The chart above is sorted by the Obama minus McCain margin, and shows the 95% confidence interval. The dot size is proportional to electoral vote.

Below I show the status of the states based on our polling categorization of each state.

Time to run for the plane.

















































































Convention Odds and Ends

Topics: Barack Obama , David Plouffe , Ed Reilly , Geoff Garin , Targeting , Twitter

First, a quick update on yesterday's post on the Obama campaign briefing. First, James Barnes of the National Journal has a write-up of the strategy spelled out by campaign manager David Plouffe yesterday that includes extended verbatim excerpts from the briefing.

In the briefing, Plouffe emphasized that when it comes to polling "all we care about is these 18 states." I had not specified those 18 states, but they are: Alaska, Colorado, Florida, Georgia, Iowa, Indiana, Michigan, Missouri, Montana, Nevada, New Hampshire, New Mexico, North Carolina, North Dakota, Ohio, Pennsylvania, Wisconsin, and Virginia.

Second, on Tuesday, I participated in a panel on polling put on by the National Journal along with Hotline editor Amy Walter and pollsters Ed Reilly, who conducts the Diageo Hotline poll and Geoff Garin, who worked for Hillary Clinton earlier this year. Video excerpts from that panel are now available at the this link.

Finally, in an hour or so, I will be going offline and heading over to Invesco Field for the evening. I've been experimenting with Twitter this week (under the handle of, what else, MysteryPollster), and will probably post some comments there tonight (with the caveat that any such "tweets" are likely to be a bit off our usual focus on polling and surveys).


POLL: Greg Smith Idaho (8/18-22)


Greg Smith and Associates
8/18-22/08; 600 LV, 4%
Mode: Live Telephone Interviews

Idaho
McCain 52, Obama 29
Sen: Risch (R) 41, LaRocco (D) 30, Rammell (i) 3


POLL: Daily Tracking (8/25-27)


Gallup Poll
8/25-27/08; 2,723 RV, 2%
Mode: Live Telephone Interviews

National
Obama 48, McCain 42

Also
"Obama Still Lags McCain as Leader, Commander in Chief"


Rasmussen Reports
8/25-27/08; 3,000 LV, 2%
Mode: IVR

National
McCain 47, Obama 47

Also
"74% of Democrats Say Convention Has Unified Them"


POLL: Hill Research Colorado (8/23-24)


Hill Research Consultants (R) /
Bob Schaffer / NRSC
8/23-24/08; 553 LV, 4.2%
Mode: Live Telephone Interviews

Colorado
Obama 43, McCain 40
Sen: Udall (D) Schaffer (R) 38


POLL: PPIC California (8/12-19)


Public Policy Institute of California
8/12-19/08; 1,047 LV, 3%
Mode: Live Telephone Interviews

California
Obama 48, McCain 39
(July: Obama 50, McCain 35)


POLL: Mason-Dixon Florida (8/25-26)


Mason-Dixon
8/25-26/08; 625 LV, 4%
Mode: Live Telephone Interviews

Florida
McCain 45, Obama 44


The Myth of "Obama Fatigue"


According to Pew's Andrew Kohut, the American electorate is suffering from "Obama fatigue." A close examination of the polling data suggests this conclusion is more of a personal opinion than one supported by the polling data.

 

Kohut came to his conclusion after first noting that the latest Pew Research Center poll in early August found Barack Obama's lead over John McCain "withering." He then noted that the same poll found more people saying they had been hearing "too much" about Obama's campaign than said that about McCain's campaign. Linking the two findings, Kohut concluded that Obama's greater news exposure over the summer "has proved a problem, not a blessing, for the Democratic candidate."

 

There are a couple of problems of data interpretation. First is the assertion of what Kohut calls a "tightening race." Pew conducted three polls - one each in June, July and August - and in those polls found Obama's lead going from eight points in June (48 percent to 40 percent), to five points in July (47 percent to 42 percent) and to just three points in early August (46 percent to 43 percent). Thus, overall, Obama's support dropped two percentage points over the summer, while McCain's increased by three. That such minor differences in the polls should be treated as a definitive trend is stunning. Even with larger-than-average sample sizes, those differences in the polls are within the polls' margins of error. In other words, even according to these polls, it's quite possible that there was no decline in Obama's lead, and perhaps even an increase. We just can't know for sure (using the 95 percent confidence level).

 

There are many other polls besides Pew that are measuring the candidates' support, but only one major media organization has conducted polls on a daily basis over this same time period. Gallup has been interviewing about 1,000 respondents each day, reporting the results on a three-day rolling average. If anyone wants to know how the campaign has changed over time, Gallup provides the best set of results. And these results do not show a linear change over the time period described by Kohut, but rather many fluctuations that defy any clear trend.

 

On June 10, Gallup reported a 6-point Obama lead, which disappeared by June 25. The lead went back to as high as six points in early July, down to one point in mid-July, up to nine points in late July, then down to zero only five days later. The lead was back up to six points on August 12, but down to one point on August 21. One can "discover" a linear three-month trend only by cherry-picking Gallup's results - but the cherry-picked trend could just as easily show an increase as a decline. In any case, the notion that "Obama fatigue" could explain all of these variations is simply not credible.

 

A second problem with data interpretation is the almost indecipherable meaning that is elicited by the question that was used to suggest Obama fatigue. The poll question Kohut cited asked whether people felt they had been hearing "too much, too little, or the right amount" about each of the campaigns. Forty-eight percent said too much about Obama's campaign, 26 percent about McCain. To be sure, that's a major gap, but what does it mean? If it means people are unhappy with hearing about Obama, and that is related to their "declining" support for him, how could Pew have found Obama's support dropping by only two percentage points, given the 22-point gap in the "fatigue" question? If that sentiment truly affected voters' support of Obama, one would expect a much greater drop.

 

More important, we know that the crucial question to explain change in support is whether the explanatory variable also shows change over the same time period. Did people become more dissatisfied from June to August with media coverage of Obama's campaign and, if so, did that increased dissatisfaction in turn cause their support to "wither"? As it turns out, Pew didn't ask that question back in June, so we don't know. Thus, statistically, we can't link dissatisfaction in the August poll with the change in support from June to August. The assertion of "Obama fatigue" is not a statistical conclusion, but an intuitive one.

 

An alternative intuitive explanation of what this question measured is that many voters may well be tired of a presidential campaign that goes on for 18 months or more - in other words, not "Obama fatigue" as much as "campaign fatigue." Dissatisfaction may have appeared to be more focused on Obama in this particular poll, because the question was asked during a time when there was more media coverage of Obama for his overseas trip. Had the question been asked at a different time, or had the pollsters tried to probe beneath the surface of this superficial question, we might have obtained a better insight into what the public was thinking.

 

Instead, we are treated to the fiction of "Obama fatigue" as a cause of a "tightening race"  - a spurious explanation of a non-event.

 

(A slightly different version of this critique was posted at HuffingtonPost.)

 


McDonald: Democratic Dissention: An Artifact of Survey Methodology?


Today's guest pollster contribution comes from Michael P. McDonald, an Associate Professor of Government and Politics in the Department of Public and International Affairs at George Mason University and a Non-Resident Senior Fellow at the Brookings Institution.

A media storyline surrounding the Democratic convention is how a sizable number of Hillary Clinton supporters are backing John McCain over Barack Obama. A recent CNN/ORC poll provides grist for the mill. Twenty-seven percent of self-identified Clinton supporters are reported backing John McCain, an increase from 16% in a similar June survey.

Yet, there are indications that something is amiss in this survey. CNN reports they interviewed 1,023 adults. The organization does not report the sub-sample size of Democrats who support Clinton, but they do provide a margin of error of this sub-sample from which we can infer the number of Clinton supporters. The reported margin of error for Democrats who support Clinton is 7.5 percentage points, which is equivalent to 171 persons assuming a simple random sample. That is 16.7% of all adults in the survey, which when applied to my 2006 voting-age population estimate of 227 million persons means that there are 38 million self-identified Clinton supporters among Democrats in the CNN/ORC poll (with a 95% confidence interval between 20.9 and 54.9 million persons).

As one might recall, Clinton received 18 million votes in the primaries. If she had received 38 million votes, she would be accepting the Democratic Party's nomination on Thursday.

The question arises, who are these 20 million or so self-identified Democrats who support Clinton who did not participate in the primaries? It is difficult to tell without analyzing the survey in depth. While there are many reasonable explanations for the discrepancy between the election and survey results, a plausible explanation consistent with the large percentage of self-identified Clinton supporters who report supporting McCain in a two-way contest against Obama is that the CNN/ORC questionnaire is worded in such a manner that elicits persons who self-report supporting McCain to report that they are a Democrat who supports Clinton for the party's nomination.

The implication is obvious: if these surveys that purport to measure Clinton supporters who will vote for McCain actually measure McCain supporters who would like to see Clinton as the Democratic nominee, the media storyline of Democratic dissention quickly unravels.


26 Million (and Counting)

Topics: Conventions , Democrats , Ratings

Following up on yesterday's post on the Nielsen ratings for the first night of the convention. A.C. Nielsen sends a press release pointing to their new blog site "with background information on political, Olympics and other viewing information."

Here's the latest:

1) Hillary's night (26.0 million viewers) had higher ratings than Michelle's night (22.3 million viewers).

2) African Americans continue to watch the convention in a higher proportion than the rest of the population (the African American rating, or percentage of the African American population watching, was 12.7 vs. a 9.0 for the population as a whole)

3) Almost five times as many people (26.0 million) watched Day Two coverage in 2008 vs. Day Two in 2004 (5.9 million) when only the cable networks covered the convention.


POLL: CNN/Time CO, NC, NM, PA (8/24-26)


CNN / Time / ORC
8/24-26/08
Mode: Live Telephone Interviews
(story)

Colorado (670 RV, 4%)
McCain 47, Obama 46
McCain 43, Obama 42, Nader 7, Barr 3, McKinney 2

Nevada (625 RV, 4%)
Obama 49, McCain 44
Obama 41, McCain 41, Nader 6, Barr 5, McKinney 3

New Mexico (659 RV, 4%)
Obama 53, McCain 40
Obama 50, McCain 36, Nader 8, Barr/McKinney 0

Pennsylvania (669 RV, 4%)
Obama 48, McCain 43
Obama 47, McCain 38, Nader 7, Barr 1


Plouffe on Obama and Polling

Topics: Barack Obama , Bounce , David Plouffe , Focus Groups , Gallup Daily , Targeting , Turnout

How is the Obama campaign using surveys and other data to guide their strategy? What do they think about national polling generally, and the Gallup Daily tracking in particular? This morning, I got an earful on both subjects at an on-the-record briefing by Obama campaign manager David Plouffe, communications director Dan Pfeiffer and campaign advisor Anita Dunn for a dozen or so editors and executives of the Atlantic Media company and the National Journal Group.

My colleague Marc Ambinder has already blogged some highlights from the session. Let me fill in a few more details that touched on the campaigns use of polling, research and targeting.

First, Ambinder reported on this exchange on national polling:

We tried to get Plouffe to react to a spate of national polls showing a tightening race.

"All we care about is these 18 states," he said. He repeated, with emphasis, that the campaign does not care about national polling. Instead, the campaign's own identification, registration and canvassing efforts provide the data he uses to determine where to invest money and resources.

Plouffe also emphasized that the internal polling the campaign does is focused on those same 18 states,** and that their real concern is not the horse race results but the "data underneath." Later, he added, "the top-line [polling data] doesn't tell you anything." Rather, they focus on who the "true undecideds" are, "how they're likely to break," and what messages will best persuade them.

The Gallup Daily tracking poll is apparently a particular sore point. When asked whether they were unhappy that the Biden announcement had not produced a bounce in national polls, Plouffe shot back: "How do you determine a bounce. . . from the Gallup Daily?" The Gallup Daily, he added is "something we don't pay attention to," he said again.

Communications director Dan Pfieffer later put it more bluntly, expressing unhappiness with the "inordinate focus on bad polling" by the media and also in the routine misinterpretation of sampling noise in the Gallup Daily poll. "The Gallup Daily is the worst thing that's happened in journalism in 10 years," he said.

Plouffe also warned against "making too big an assumption" based on focus groups when asked about the Frank Luntz group of undecided voters that received a fair amount of attention this week. "We certainly don't use [focus] groups to make assessments of swing voters," he said. They conduct focus groups, mostly "to hear people talk" about the issues and candidates, but when it comes to identifying "true undecided" voters, their emphasis is on quantitative data, including traditional surveys and data on registration and vote history collected from lists and supplemented with information gleaned through direct voter contact.

I asked about Marc Ambinder's report of the "data" collected by the Obama campaign Monday night that left them with a "high degree of confidence" that Michelle Obama's speech went over well in their 18 target states. Marc had inferred that "the campaign ran several focus groups" Monday might, but I'm skeptical given the logistical challenge of doing traditional focus groups in 18 states in one night. My guess is that they ran some sort of online test, and asked Plouffe if he could add more detail:

That information "will have to be a mystery," enjoying the play on my nomme de Internet. He did say that their efforts to monitor undecided voters features "not through the traditional methods of quantitative and qualitative research.  We also have hundreds of thousands of contacts [made] every night."

Much of the briefing covered specifics on the focus on turnout by the Obama campaign and their massive effort to "adjust the electorate" to their benefit. He cited several examples, including Florida where he claimed that roughly 600,000 African Americans that were registered but did not vote in 2004, with more than half of that group coming from African Americans under 40 years of age. "If we just execute on turnout" in Florida, he said, "we're going to be bumping up on our win number." They also believe they can keep states like Virginia and North Carolina competitive if they "blow the doors off turnout."

The briefing included much more that my National Journal and Atlantic Media colleagues will be reporting on later today and this week, and I will try to add links here as they become available. I'm also likely to say more at some point about the Obama campaign's overall approach to research and strategy based on these comments.

Finally, please note that the verbatim quotations above are from my notes. We are hoping to post a full transcript later in the week.  [Update/Correction:  With a transcript in hand, I have corrected a few minor wording errors.  In the original version, I erroneously quoted Dan Pfeiffer describing the Gallup Daily as "the worst thing that's happened in journalism in 20 years" -- he actually said 10 years.]    

Update - Here's a quick response via email from Gallup's Frank Newport: 

These are the same types of sentiments that have been expressed since George Gallup's first presidential polls in 1936.  Campaigns like to control the narrative, and don't like outside intrusion in their story lines. Bottom line:  The American public is vastly interested, and always has been, in where a presidential race stands during a campaign.  Gallup (and others) can help provide a scientific answer to that question, using careful methodology and deliberate analysis.  Without independent polling, the public would be reliant on campaign operatives' self-promoting insights on where the race stands, or on journalists' guesses.  And, of course, polling provides a vast array of insights into the dynamics and currents of a campaign and represents the voters' views, thoughts, and wishes.

**Update 2: The 18 states that the Obama campaign is focusing on are: Alaska, Colorado, Florida, Georgia, Iowa, Indiana, Michigan, Missouri, Montana, Nevada, New Hampshire, New Mexico, North Carolina, North Dakota, Ohio, Pennsylvania, Wisconsin, and Virginia.



POLL: Strategic Vision Florida (8/22-24)


Strategic Vision (R)
8/22-24/08; 1,200 LV, 3%
Mode: Live Telephone Interviews

Florida
McCain 49, Obama 42, Barr 1
(June: McCain 49, Obama 41, Barr 1)


POLL: Brown University Rhode Island (8/18-20)


Brown University
8/20/08; 548 RV, 4%
Mode: Live Telephone Interviews

Rhode Island
Obama 51, McCain 30


McCain, Obama and Clinton Favorability


FavBOJMHC.png

A little interesting movement in views of the candidates has taken place since the end of the primaries in June. All three candidates, McCain, Obama and Clinton, have seen rises in their favorable ratings and an initial decline in unfavorable views though with a slight upturn recently. McCain and Obama are enjoying essentially identical ratings, with 60% favorable and only 35% unfavorable. Even after a significant amount of negative portrayals of him in RNC and McCain ads, Obama's rating has risen over the summer, and so has McCain's. (According to the Wisconsin Advertising Project, which monitored and coded all 100,000 ad airings in June and July, one third of McCain's ads contained negative information about Obama and 100% of RNC ads were negative. In the same two months, 10% of Obama's ads mentioned McCain.)

Whatever happens after the conventions, both candidates enjoy an enviable standing with voters as attractive figures instead of a pair of lesser evils. The fall capaign may alter this, but even after a hard fought primary season the nominees remain attractive figures.

Meanwhile, Senator Clinton has also enjoyed an upturn in favorable ratings and a decline in unfavorable ratings since the end of the primary season. While improved, Clinton remains a more polarizing figure than either McCain or Obama, with slightly lower favorable but noticeably higher negative ratings.

Senator Clinton is far more popular among Democrats than among either Independents or (especially) Republicans. In that sense, her speech to the Democratic Convention last night was an example of speaking primarily to the party and her supporters, rather than to the broader public. The contast between former Virginia governor and now Senate candidate Mark Warner's speech and Clinton's is a good example of this difference. Warner stressed unifying themes and appeals across political groups, which was greated warmly but which fell short of electrifying the Democratic delegates. In contrast, Clinton played to the party and produced a predictably enthusiastic response within the DNC convention hall. Conventions contain both elements. Monday, the party celebrated Sen. Kennedy's life and family legacy, primarily an inside the family affair, perhaps touching some independents but not likely to attract Republicans. In contrast Michelle Obama's speech could have easily been given at the Republican convention, with its themes of family, hard work, pulling oneself up from working class circumstances. Hers was a speech designed to reach out beyond the party.

The one remaining question from the Clinton speech is whether her supporters also resepect her enough to follow her lead. For Clinton to be a power in the party includes the requirement that she be able to deliver her supporters for Obama. If any significant number of her supporters refuse to be delivered, they reduce her status as a result. This is hard to judge from the cable news coverage, who can easily find individual delegates willing to say they are unpersuaded. But what effect the Clinton speech has with her supporters outside the convention hall will be critical.


22 Million


The Page is linking to a report from the LA Times blog on the Nielsen ratings from last night:

The opening night of the Democratic National Convention drew more than 22 million viewers, a 20% larger audience than in 2004, according to Nielsen Media Research.

If you want even more detail on the numbers (and I know you do) check out this post on the very helpful blog, TV By The Numbers.

To keep all of these numbers in perspective, consider that 22 million is double the 10.7 million that watched ABC's broadcast of the April 16 Clinton-Obama debate in Pennsylvania and five to twenty times the audience size of the many debates broadcast in 2007.

On the other hand, 22 million is still a far cry from the 62 million that watched the first Bush-Kerry debate in 2004.

These numbers show us that while the conventions are the most watched political events so far this cycle, they are still not quite the voters-as-jury experience that we sometimes assume. The news coverage that excerpts speeches and convention "moments" reaches a far bigger cumulative audience. Those of us interested in measuring the impact of the debates need to allow time for Americans to view that coverage, absorb it and sleep on it for a few days.

In other words, be patient and stop worrying about the "bounce" (or lack thereof) in daily tracking. Conventions matter, but the response we are interested is not necessarily instant.


Hillary's Opportunity


My second National Journal column for the week (which will appear in tomorrow's Convention Daily) will be posted within the next few hours. Since it is all about the opportunities presented by Hillary Clinton's speech tonight, I'll post a key block quote now and add the link later.  Update:  the full column is now live.

The gist is that I disagree with the "Hillary can't win," damned if she does, damned if she doesn't them of Marie Cocco's column in this morning's Washington Post (echoed to some extent by Todd, et. al. in FirstRead). I think the speech presents Clinton with a huge opportunity, both for her own long term interests and for the Obama-Biden ticket. I make the case with survey data in the column. While an Obama-Clinton ticket would have come with risks to offset benefits, the same cannot be said for tonight's speech. Quoting myself:

And the decision by the McCain campaign to release (if not air) three different television advertisements this week invoking Clinton's criticisms of Obama during the primaries provides her with a huge tactical opportunity to create one of the convention's most memorable moments.

"I'm Hillary Rodham Clinton, and I do not approve of that message," she told the New York state delegation yesterday. In her speech, she can do more. I am not a speechwriter, but the "truth hurts" tagline of the first of these spots seems like an obvious opening for a riff on the records of McCain and President Bush.

We'll see how it turns out.

PS: Nate Silver made a very similar point about the Clinton-quoting McCain ads earlier in the week:

I could see the ad being very effective. But it also tosses a big softball to Hillary Clinton, who will speak to a national audience on Tuesday. The risk to the Republicans can be summarized in five words: "Shame on You, John McCain". A finger-wagging, how-dare-you moment by either of the Clintons at the convention -- but especially Hillary -- could be both effective and therapeutic, especially when coupled with a reminder that McCain voted against measures like SCHIP (and voted to impeach her husband).

I prefer big "hanging curve ball," but I'll defer to the baseball guy. 

PPS:  I'm catching up on my RSS feed while listening to the speeches.  This post yesterday from Marc Ambinder seems relevant to what Clinton can help accomplish (emphasis added):

They are, yes, Hillary supporters, but a certain type of Hillary supporters: mainly white voters without college degrees. Ron Brownstein has noted that in four polls taken before the convention, Obama sits at 38% with this group.  These voters, as pollster Stan Greenberg's new data shows, have a panoply of concerns. Unquestionably, some are racist. But a majority of them worry about Obama's credentials, his liberal positions on national security issues, and whether he truly understands their economic insecurities.

It is much easier to convince these voters to vote for Obama when they see Obama as the antidote to the Bush presidency, and when they see McCain as a Bush Republican. SO -- you will hear and see speaker after speaker portray McCain as a Bush Republican.  Polling shows that even when recalcitrant Democrats learn about Obama's middle class roots, they're still skeptical. It is MUCH harder to convince them to vote for Obama because they LIKE him. It is much easier to convince them to vote for Obama because they think McCain represents a continuation of President Bush's policies. (Obama's campaign has polling data suggesting that an unusually large number of pro-choice Democrats don't know that McCain is pro-life.)


POLL: Daily Tracking (8/23-25)


Gallup Poll
8/23-25/08; 2,684 RV, 2%
Mode: Live Telephone Interviews

National
McCain 46, Obama 44


Rasmussen Reports
8/23-25/08; 3,000 LV, 2%
Mode: IVR

National
McCain 46, Obama 46


POLL: Diageo/Hotline National (8/18-24)


Diageo / The Hotline
8/18-24/08; 1,022 RV, 3%
Mode: Live Telephone Interviews

National
Obama 44, McCain 40
(June: Obama 44, McCain 42)


POLL: PPP North Carolina (8/20-23)


Public Policy Polling (D)
8/20-23/08; 904 LV, 3.3%
Mode: IVR

North Carolina
McCain 45, Obama 42, Barr 4
(July: McCain 47, Obama 44, Barr 3)

Sen: Hagan (D) 42, Dole (R-i) 39, Cole (L) 5
(July: Dole 40, Hagan 40, Cole 4)

Gov: Perdue (D) 43 , McCrory (R) 38, Munger (L) 4
(July: Perdue 46, McCrory 37, Munger 6)


POLL: Quinnipiac Florida, Ohio, Pennsylvania (8/17-24)


Quinnipiac University
8/17-24/08
Mode: Live Telephone Interviews

Florida (1,069 LV, 3%)
McCain 47, Obama 43
(July: Obama 46, McCain 44)

Ohio (1,234 LV, 2.8%)
Obama 44, McCain 43
(July: Obama 46, McCain 44)

Pennsylvania (1,234 LV, 2.8%)
Obama 49, McCain 42
(July: Obama 49, McCain 42)


POLL: Kitchens Florida (8/18-21)


Florida Chamber of Commerce /
The Kitchens Group (D)
8/18-21/08; 605 LV, 4%
Mode: Live Telephone Interviews

Florida
McCain 42, Obama 39


Measuring the Bounce


Here's the link Convention Daily edition of my National Journal column for today, which covers the difficulty pollsters will have measuring the "bounce" that Barack Obama gets this week. The short version: There will be many polls this week but no way to measure the bump or bounce that is comparable to past years. So it's probably best to dispense with metaphysical comparisons to years past and just focus on what surveys tell us about what voters are learning and what conclusions they are coming to.

Incidentally, I will be writing two Conventional Daily columns for the National Journal this week (the second will appear on Wednesday morning), and two next week. In exchange for this contribution, I get to be on hand in Denver this week and in Minneapolis next week.

I've been traveling most of today and just got to our work space. Soon I'll be off to wander the hall, and seek out pollsters and their wisdom...


POLL: Rasmussen Texas (8/21)


Rasmussen Reports
8/21/08; 500 LV, 4.5%
Mode: IVR

Texas
McCain 54, Obama 44
(July: McCain 52, Obama 44)

Sen: Cornyn (R-i) 52, Noriega (D) 38
(July: Cornyn 50, Noriega 39)


POLL: SurveyUSA Issues in 15 States (8/15-17)


SurveyUSA
8/15-17/08; Registered Voters
Mode: IVR

Among other questions:

    Which candidate for President do you think will do a better job handling the economy?

    Which candidate for President do you think will do a better job handling the situation in Iraq?

New Mexico
Economy: Obama 49, McCain 38
Iraq: Obama 47, McCain 41

Minnesota
Economy: Obama 46, McCain 44
Iraq: McCain 46, Obama 43

Missouri
Economy: Obama 46, McCain 43
Iraq: McCain 49, Obama 41

Wisconsin
Economy: McCain 45, Obama 43
Iraq: McCain 50, Obama 41

Ohio
Economy: Obama 49, McCain 40
Iraq: Obama 45, McCain 44

Virginia
Economy: McCain 52, Obama 41
Iraq: McCain 55, Obama 36

Iowa
Economy: Obama 47, McCain 43
Iraq: McCain 49, Obama 42


POLL: Zogby National (8/23-24)


Zogby Interactive
8/23-24/08; 2,248 LV, 2.1%
Mode: Internet

National
Obama 46, McCain 44
(8/14: Obama 44, McCain 42)

Obama 45, McCain 42, Barr 4, McKinney 1, Nader 1
(8/14: Obama 43, McCain 40, Barr 6, Nader 2)


POLL: EPIC/MRA Michigan (8/18-21)


EPIC-MRA /
The Detroit News
8/18-21/08; 600 LV, 4%
Mode: Live Telephone Interviews

Michigan
Obama 43, McCain 41, Nader 3, Barr 1
(July: Obama 43, McCain 41, Nader 3, Barr 2

Sen: Levin (D-i) 59, Hoogendyk (R) 27
(July: Levin 58, Hoogendyk 32)


POLL: Columbus Dispatch Ohio (8/12-21)


Columbus Dispatch
8/12-21/08; 2,102 RV, 2.2%
Mode: Mail

Ohio
McCain 42, Obama 41, Nader 1, Barr 1


POLL: Suffolk Colorado (8/21-24)


Suffolk University
8/21-24/08; 450 LV, 4.6%
Mode: Live Telephone Interviews

Colorado
Obama 44, McCain 39, Nader 2, Barr 2


POLL: CNN National (8/23-24)


CNN / ORC
8/23-24/08; 909 RV, 3.5%
Mode: Live Telephone Interviews

National
Obama 47, McCain 47
(July: Obama 51, McCain 44)

Obama 44, McCain 44, Nader 4, Barr 2, McKinney 1
(July: Obama 46, McCain 42, Nader 6, Barr 3, McKinney 1)


How Pollsters Affect Poll Results

Topics: House Effects
























Who does the poll affects the results. Some. These are called "house effects" because they are systematic effects due to survey "house" or polling organization. It is perhaps easy to think of these effects as "bias" but that is misleading. The differences are due to a variety of factors that represent reasonable differences in practice from one organization to another.

For example, how you phrase a question can affect the results, and an organization usually asks the question the same way in all their surveys. This creates a house effect. Another source is how the organization treats "don't know" or "undecided" responses. Some push hard for a position even if the respondent is reluctant to give one. Other pollsters take "undecided" at face value and don't push. The latter get higher rates of undecided, but more important they get lower levels of support for both candidates as a result of not pushing for how respondents lean. And organizations differ in whether they typically interview adults, registered voters or likely voters. The differences across those three groups produce differences in results. Which is right? It depends on what you are trying to estimate-- opinion of the population, of people who can easily vote if the choose to do so or of the probable electorate. Not to mention the vagaries of identifying who is really likely to vote. Finally, survey mode may matter. Is the survey conducted by random digit dialing (RDD) with live interviewers, by RDD with recorded interviews ("interactive voice response" or IVR), or by internet using panels of volunteers who are statistically adjusted in some way to make inferences about the population.

Given all these and many other possible sources of house effects, it is perhaps surprising the net effects are as small as they are. They are often statistically significant, but rarely are they notably large.

The chart above shows the house effect for each polling organization that has conducted at least five national polls on the Obama-McCain match-up since 2007. The dots are the estimated house effects and the blue lines extend out to a 95% confidence interval around the effects.

The largest pro-Obama house effect is that of Harris Interactive, at just over 4 points. The poll most favorable to McCain is Rasmussen's Tracking poll at just less than -3 points. Everyone else falls between these extremes.

Now let's put this in context. We are looking at effects on the difference between the candidates, so that +4 from Harris is equivalent to two points high on Obama and two points low on McCain. Taking half the estimated effect above gives the average effect per candidate. The average effects are at most 2 points per candidate. Not trivial, but not huge.

Estimating the house effect is not hard. But knowing where "zero" should be is very hard. A house effect of zero is saying the pollster perfectly matches some standard. The ideal standard, of course, is the actual election outcome. But we don't know that now, only after the fact in November. So the standard used here is the house effect relative to our Pollster Trend Estimate. If a pollster consistently runs 2 points above our trend, their house effect would be +2.

The house effects are calculated so that the average house effect is zero. This doesn't depend on how many polls a pollster conducts. And it doesn't mean the pollster closest to zero is the "best". It just means their results track our trend estimate on average. That can also happen if a pollster gyrates considerably above and below our trend, but balances out. A nicer result is a poll that closely follows the trend. But either pattern could produce a house effect near zero. For example, Democracy Corps and Zogby have very similar house effects near -1. But look at their plots below and you see that Democracy Corps has followed our trend quite closely, though about a point below the trend. Zogby has also been on average a point below trend, but his polls have shown large variation around the trend, with some polls as near-outliers above while others are near outliers below the trend. The net effect is the same as for Democracy Corps, but the variability of Zogby's results is much higher.

Incidentally, the Democracy Corps poll is conducted by the Democratic firm of Greenberg Quinlan Rosner Reserch in collaboration with Democratic strategist James Carville. Yet the poll has a negative house effect of -1. Does this mean the Democracy Corps poll is biased against Obama? No. It means they use a likey voter sample, which typically produces modestly more pro-Republican responses than do registered voter or adult samples. Assuming that the house effect necessarily reflects a partisan bias is a major mistake.

How can you use these house effects? Take a pollster's latest results and subtract the house effect from their reported Obama minus McCain difference. That puts their results in the same terms as all others, centered on the Pollster.com Trend Estimate. This is especially useful if you are comparing results from two pollsters with different house effects. Removing those house differences makes their results more comparable.

What impact do house effects have on our Pollster.com Trend Estimate? A little. Our estimator is designed to resist big effects of any single pollster, but it isn't infallible, especially when some pollsters do far more polls than others or when one pollster dominates during some small period of time. We can estimate house effects, adjust for these, and reestimate our trend with house effects removed. The result runs through the center of the polls, but doesn't allow the number of polls done by an organization to be as influential.

The results are shown in the chart below. The blue line is our standard estimator and the red line is the estimate with house effects removed. Without house effects the current trend stands at +2.0 while ignoring house effects produces an estimate of +1.7. A little different, but given the range of variability across polls and the uncertainty as to where the race "really" stands, this is not a big effect.























The impact of house effects isn't always this small. Looking back along the trend we see that the red and blue lines diverged by as much as 1 point in late June, an effect due significantly to the large number of Rasmussen and Gallup tracking polls during that time and few polls with positive house effects in that period. A smaller but still notable divergence occurred in late February and early March.

The bottom line is that there are real and measurable differences between polling organizations, but the magnitude of these effects is considerably less than some commentary would suggest. Many of the house effect estimates above are not statistically different from zero. Even ignoring that, the range of effects is rather small, though of course in a tight race the differences may be politically important. Finally, the effects on our Pollster.com Trend Estimate is detectable but does not lead to large distortions, even if we can see some noticeable differences at some times.

The charts below move though all the pollsters and plots their poll results compared to the standard trend and the trend removing house effects. Pollsters with fewer than 5 polls are all lumped together as "Other" pollsters. Once they get to our minimum number of polls, we'll have house effects for them too.























POLL: Mason-Dixon Southwest States (8/13-15)


Mason-Dixon / The Denver Post /
The Las Vegas Review-Journal / The Salt Lake Tribune
8/13-15/08; 400 LV, 5% (in each state)
Mode: Live Telephone Interviews
(results)

Arizona
McCain 47, Obama 41

Colorado
Obama 46, McCain 43

Nevada
McCain 47, Obama 41

New Mexico
McCain 45, Obama 41

Utah
McCain 62, Obama 23
Gov: Huntsman (R-i) 73, Springmeyer (D) 9

Wyoming
McCain 62, Obama 25


POLL: PPP Virginia (8/20-22)


Public Policy Polling (D)
8/20-22/08; 1,036 LV, 3%
Mode: IVR

Virginia
Obama 47, McCain 45
(July: Obama 46, McCain 44)

Sen: Warner (D) 55, Gilmore (R) 32
(July: Warner 57, Gilmore 32)


POLL: Quinnipiac Colorado (8/15-21)


Quinnipiac University
8/15-21/08; 1,060 LV, 3%
Mode: Live Telephone Interivews

Colorado
McCain 47, Obama 46
(July: McCain 46, Obama 44)


POLL: ABC/Post National (8/19-22)


ABC News /
Washington Post
8/19-22/08; 916 RV, 3%
Mode: Live Telephone Interviews
(ABC story, results; Post story, results)

National
RV: Obama 49, McCain 43
(July: Obama 50, McCain 42)

LV: Obama 49, McCain 45
(July: Obama 49, McCain 46)


 

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR