Pollster.com

Articles and Analysis

 

Plouffe on Obama and Polling

Topics: Barack Obama , Bounce , David Plouffe , Focus Groups , Gallup Daily , Targeting , Turnout

How is the Obama campaign using surveys and other data to guide their strategy? What do they think about national polling generally, and the Gallup Daily tracking in particular? This morning, I got an earful on both subjects at an on-the-record briefing by Obama campaign manager David Plouffe, communications director Dan Pfeiffer and campaign advisor Anita Dunn for a dozen or so editors and executives of the Atlantic Media company and the National Journal Group.

My colleague Marc Ambinder has already blogged some highlights from the session. Let me fill in a few more details that touched on the campaigns use of polling, research and targeting.

First, Ambinder reported on this exchange on national polling:

We tried to get Plouffe to react to a spate of national polls showing a tightening race.

"All we care about is these 18 states," he said. He repeated, with emphasis, that the campaign does not care about national polling. Instead, the campaign's own identification, registration and canvassing efforts provide the data he uses to determine where to invest money and resources.

Plouffe also emphasized that the internal polling the campaign does is focused on those same 18 states,** and that their real concern is not the horse race results but the "data underneath." Later, he added, "the top-line [polling data] doesn't tell you anything." Rather, they focus on who the "true undecideds" are, "how they're likely to break," and what messages will best persuade them.

The Gallup Daily tracking poll is apparently a particular sore point. When asked whether they were unhappy that the Biden announcement had not produced a bounce in national polls, Plouffe shot back: "How do you determine a bounce. . . from the Gallup Daily?" The Gallup Daily, he added is "something we don't pay attention to," he said again.

Communications director Dan Pfieffer later put it more bluntly, expressing unhappiness with the "inordinate focus on bad polling" by the media and also in the routine misinterpretation of sampling noise in the Gallup Daily poll. "The Gallup Daily is the worst thing that's happened in journalism in 10 years," he said.

Plouffe also warned against "making too big an assumption" based on focus groups when asked about the Frank Luntz group of undecided voters that received a fair amount of attention this week. "We certainly don't use [focus] groups to make assessments of swing voters," he said. They conduct focus groups, mostly "to hear people talk" about the issues and candidates, but when it comes to identifying "true undecided" voters, their emphasis is on quantitative data, including traditional surveys and data on registration and vote history collected from lists and supplemented with information gleaned through direct voter contact.

I asked about Marc Ambinder's report of the "data" collected by the Obama campaign Monday night that left them with a "high degree of confidence" that Michelle Obama's speech went over well in their 18 target states. Marc had inferred that "the campaign ran several focus groups" Monday might, but I'm skeptical given the logistical challenge of doing traditional focus groups in 18 states in one night. My guess is that they ran some sort of online test, and asked Plouffe if he could add more detail:

That information "will have to be a mystery," enjoying the play on my nomme de Internet. He did say that their efforts to monitor undecided voters features "not through the traditional methods of quantitative and qualitative research.  We also have hundreds of thousands of contacts [made] every night."

Much of the briefing covered specifics on the focus on turnout by the Obama campaign and their massive effort to "adjust the electorate" to their benefit. He cited several examples, including Florida where he claimed that roughly 600,000 African Americans that were registered but did not vote in 2004, with more than half of that group coming from African Americans under 40 years of age. "If we just execute on turnout" in Florida, he said, "we're going to be bumping up on our win number." They also believe they can keep states like Virginia and North Carolina competitive if they "blow the doors off turnout."

The briefing included much more that my National Journal and Atlantic Media colleagues will be reporting on later today and this week, and I will try to add links here as they become available. I'm also likely to say more at some point about the Obama campaign's overall approach to research and strategy based on these comments.

Finally, please note that the verbatim quotations above are from my notes. We are hoping to post a full transcript later in the week.  [Update/Correction:  With a transcript in hand, I have corrected a few minor wording errors.  In the original version, I erroneously quoted Dan Pfeiffer describing the Gallup Daily as "the worst thing that's happened in journalism in 20 years" -- he actually said 10 years.]    

Update - Here's a quick response via email from Gallup's Frank Newport: 

These are the same types of sentiments that have been expressed since George Gallup's first presidential polls in 1936.  Campaigns like to control the narrative, and don't like outside intrusion in their story lines. Bottom line:  The American public is vastly interested, and always has been, in where a presidential race stands during a campaign.  Gallup (and others) can help provide a scientific answer to that question, using careful methodology and deliberate analysis.  Without independent polling, the public would be reliant on campaign operatives' self-promoting insights on where the race stands, or on journalists' guesses.  And, of course, polling provides a vast array of insights into the dynamics and currents of a campaign and represents the voters' views, thoughts, and wishes.

**Update 2: The 18 states that the Obama campaign is focusing on are: Alaska, Colorado, Florida, Georgia, Iowa, Indiana, Michigan, Missouri, Montana, Nevada, New Hampshire, New Mexico, North Carolina, North Dakota, Ohio, Pennsylvania, Wisconsin, and Virginia.


 

Comments

I've been complaining for a while about the over-interpretation of movement in the daily tracking polls, too. But I wonder if the Obama campaign might have had kinder words for Gallup given that Obama jumped 3 percentage points to regain a 1 point lead in today's tracking poll. Obama must have had a good night of polling last night. And while Gallup says this is an indication that Obama may be developing a convention bounce, as David Pfieffer says, there is no way to distinguish today's movement (or even yesterday's movement) from statistical noise.

____________________

jatchwa:

Which 18 states?

____________________

jradMIT:

Just a question does anybody have thoughts on the possiblity of a strong hurricane being in the gulf this weekend and possibly making landfall early next week on the gulf coast, and its effect on the media coverage for both sides of the aisle?

____________________

Jeff:

Surely they include OH, MI, VA, PA, CO, NM, NM, NH, NV, IA, MO, WV, NC, DE, MN -- not sure about the other 3.

____________________

Mark Lindeman:

@Michael: I was going to page you to ask about the roughly 600,000 Florida African-Americans who were registered but didn't vote in 2004. Is that right?

I see only 1.22 million blacks (plus some "other"s and "not given"s) on the Florida rolls in 2004, period. (http://election.dos.state.fl.us/voter-registration/statistics/pdf/2004/2004genRace.pdf) I haven't spotted turnout by race, but I doubt it was so low. (Probably I have the answer to this in one of your papers.)

____________________

My response to Frank Newport:

I'd like much more if your analysis focused on longer-term trends than short-term movement. All too often, the Gallup and Rassmussen commentary connects recent poll movement with a campaign event. More recently, Gallup seems to have toned it down a bit, but even today's write-up is slipping back into the bad habit of interpreting a single day's worth of polling to be an early indicator of an Obama bounce.

The problem with these daily tracking polls is the same problem with cable television campaign coverage. Needing to fill the vacuum by posting day after day, these polling firms feel compelled to offer equally vacuous commentary. Just because a critique is coming from the Obama campaign does not mean that it is without merit.

____________________

@Mark:

The short answer is yes.

To answer your question, I ran a query into the Feb. 2005 Florida voter file, which is the first available statewide voter file with vote history (well, almost, some counties' voting history comes from a later file and I had to get Lee County separately).

Florida is one of the few states that maintains a record of a registrant's race. According to my query, there were 555,399 African-Americans who were registered but did not vote (798,537 voted). The National Voter Registration Act of 1993 (aka Motor Voter) requires that if a local election office mails a registrant a verification of registration and the voter fails to respond to the letter or vote for two federal elections, then they may be removed or "purged" from the voter file. So, I suspect a good percentage of those 555,399 who were registered but did not vote are actually no longer living at their address, what is commonly referred to as "deadwood." The upper bound on turnout efforts would thus be less than Plouffe indicates from his citation of registrants who did not vote.

Now, this is my voter file which I have taken pains to create as the best snapshot of the 2004 election as possible. Plouffe may be working from a different file, perhaps one from the close of registration that had not been yet purged following the 2004 election. That file may have a number closer to 600,000. But we're in the same ballpark, so I think his statement is generally correct in terms of the numbers.

____________________

thoughtful:

Mark I was surprised that he was so candid. Simply, the campaign are identifying their guys and have major investments in getting them to the polls. The whole election turns on AAs: 18-29 and white women its as simple as that in 18 states!

____________________

I'll have to agree with Michael McDonald. Frank Newport has his own bias, which is that he wants people to respect and buy his product, or think it's unique. Critiques from either side should be judged on their own merits. In this case, Plouffe and Pfeiffer's statements have a very good statistical basis.

To reiterate something well known to many readers of this site: when a daily sample of 1000 likely voters has a 3-point margin of error, the difference between two such samples has a combined error that is 1.4-fold higher, in this case about 4.5%. Measuring perceived swings using one organization's data is a cheap source of news, but is a form of hucksterism.

The averaging of multiple polls does better. For instance, the last five national matchups on Pollster.com have a total of nearly 9000 respondents, giving an effective margin of error of about 1%. There's the added advantage of averaging out house effects. And of course the Pollster.com hosts have their own, more sophisticated recipe for reducing uncertainty. However, even that approach can only detect a swing of at least a few percentage points.

One can get an even larger sample size by combining all the juicy state data, which is basically what I do at the Princeton Election Consortium (as does FiveThirtyEight and others). The theoretical resolving power of all those combined respondents is well under a percentage point. However, then there's the problem of poll recency. For day-to-day tracking, I think the wiggly red and blue lines at Pollster.com are best. (Sorry to get technical there at the end.)

____________________

Perm Dude:

The Gallup response seems pretty weak, IMO. Talking about giving Americans what they are interested in answers a different question than the one the Obama campaign raises. Americans are interested in Jen and Brad (for instance), but does responding to this interest answer the question of whether daily national tracking is meaningful? Until we have a national election (as opposed to 50 state ones), I would have to say "no."

____________________

Correction:

The National Voter Registration Act of 1993 (aka Motor Voter) requires that if a local election office mails a registrant a verification of registration and the voter fails to respond to the letter AND vote for two federal GENERAL elections, then they may be removed or "purged" from the voter file.

I don't want to send out bad information about federal law. And if you're not registered and you are enough of a political junkie to read this webpage's comments section, then what are you waiting for?

____________________

cinnamonape:

Apparently Florida turnout by AA's (50%) was lower than nationwide in 2004...where it was about 60% of those registered.

In addition many AA's were taken off the voting rolls because they had names identical with registered ex-felons, or because of a recent move from prior registration. There has been a major registration effort by Democrats that has been quite successful in increasing registration by AA's, in addition to Gov. Crist signing legislation allowing voting by a variety of ex-felons (non-violent individuals who are no longer on parole) to vote. While this could actually increase the number of eligible white and Hispanic voters, such a restriction previously impacted young male AA's disproportionately. It's unclear whether this will benefit Republicans or Democrats more. In numbers it might assist McCain (if males from those demographics actually register and vote according to prior patterns)...but if AA's preferentially vote and do so for Obama, then it might more than offset that.

____________________

cinnamonape:

Sam...the fact that some pollsters use automated polling systems that sample daily (and use three day rolling samples) allows them to produce many poll outcomes that would perhaps skew any multiple meta-polling towards their results.

BTW when Rassmussen does a poll with N=1000 that means they actually are only sampling 333 per day over three days. Then they recycle two of those days to recreate a N=1000 outcome for their "new" daily poll.

____________________

cinnamonape, those are good points. I used 1000 as an example because the 3-day Gallup average on this site includes about 3000 respondents. The Rasmussen sample is shockingly small.

Your point about rolling averages suggests that results could be cherry-picked to generate all kinds of stories. You're not implying that a polling or media organization does that, are you? Ha.

The nuggets in Ambinder's piece are quite interesting. Plouffe's statements are empirically driven - in basic agreement with what I'm seeing in the numbers. It may be spin, but it is data-based.

____________________

cinnamonape:

"Jeff: Surely they include OH, MI, VA, PA, CO, NM, NM, NH, NV, IA, MO, WV, NC, DE, MN -- not sure about the other 3."

NM twice? Do you mean North Dakota? And WV? I very much suspect that Florida's being watched, after all, why would Plouffe mention it?

And perhaps Missouri and Wisconsin. They may also be interested in Georgia, simply because this state could pose problems if AA turnout was high while Republican turnout was depressed. They might not win the state, but they could be following it in order to see if putting money there might have some "coat-tail" effects. There could be other states in this category, where Obama might not win the State, but "party building activities" increasing registration and turnout could put Democrats into Congress or a Governorship.

The Obama Campaign is looking at specific demographics and what activities will increase the turnout in those areas.

____________________

dickdata:

The eighteen states should be (sort of east to west) NH, VA, NC, GA, FL, PA, OH, IN, MI, WI, MN, IA, MO, CO, NM, ND, NV, and MT. I have heard that the 10-day Gallup is a straight line and the 1-day Gallup is too erratic, so the 3-day was selected because it looks believable but shows enough movement to generate news stories. So you get stories like you do for the Dow: "McCain went up today because of the ads he has been running" or "McCain went down today in spite of the ads he has been running". If you want the subject of the headline to be Obama, just change the name and direction for THE SAME STORY.

____________________

Clint Cooper:

Jeff they do not include DE or WV.

Here are the 18 states:

ND, MT, CO, NM, NV, IA, OH, PA, MI, FL, VA, NC, GA, AK, MI, MO, NH, and probably MN or WI

____________________

The top 18 states in the Battle Rankings at election-projection.net are:

OH, VA, MI, CO, MN, NM, FL, PA, NV, NH, MT, MO, OR, ND, AK, NC, IA, IN

The election-projection.net Battle Rankings are based on the conditional probability of each state swinging the outcome of the election.

GA is not on that list, but based on public statements, I would guess the Obama campaign is watching GA and not OR.

____________________

rustyaustin:

In 2008, any poll that doesn't talk to people with cellphones and no landline cannot possibly be accurate, and I suspect those people favor Obama by at least 10 points.

____________________

mtorrens:

Michael is right. It's not the collection and presentation (or even an honest statistical analysis) of data that's the problem, it's the need for "vacuous commentary" [I'm sorry, I mean interpretive analysis] as an integral part of the business model(whether your product is polling data, cable news, or investment advice on the S&P 500). If you want to sell something today, an answer like: "I need more information to know what the trending is" doesn't cut it.

It's this overwhelming need to provide faux understanding/correlation/causation, every minute of every hour of every day, that helps to create the Schrodinger's cat problem we face - when "presentation" of the information itself helps to structure, shape, and (yes, even influence)the physiology of the participants that are supposed to be the objects of observation.

____________________

Clint Cooper:

I forgot about IN. He's probably following that one, since he has 20 campaign offices there.

____________________

Clint Cooper:

Another thing is this: Why would anyone follow the Gallup tracking or any national tracking. The states elect Presidents, not the popular vote. That's why examining the cross-tabs of the battleground polls is particularly relevant.

____________________

faithhopelove:

Looking at where Obama has been on the air, Clint's list of states looks about right.

rustyaustin:
Here's some info on the impact of not polling cell-phone only voters (it hurts Obama's numbers by 2-3%):
http://pewresearch.org/pubs/901/cell-phones-polling-election-2008

____________________

Jeff:

Many thanks to those who expanded upon/corrected my off-the-cuff guess of the 18 states.

____________________

cinnamonape:

Re. the cell phone data. A lot of the Pew Study is based onm the premise that a) there is a 3:1 land-line vs. cell-phone sample (that is based upon Dept. of Commerce data "for" 2007 that actually is several years old); and b) that cell phone users need to be devalued in their LV estimates because only about 54% said they absolutely intended to vote (vs. probably). If they had included the latter category they would have matched the % of absolutely+probably that landline users had attained.

The Pew Study indicated that when these modifications were done that the land-line only vs. the COMBINED (3:1 mix) landline+cellphone sample had a 3-4% difference. But if one looks at the crosstabs there are vast differences between cell-phone only (61% Obama vs. 32% McCain) and land-line only (46%O-41%Mc) samples.

That's also true of swing vote data. When one ranks those that are certain Obama vs. McCain voters (rather than likely) the cell-phone users are 46% to 18% more certain to select Obama vs. McCain, with 36% in flux. While Obama loses 25% [of a base of 61%] support (i.e 41% soft), McCain's loss of 14% [from a base of 32%] is about the same (43% soft). But Obama starts with a much higher level of support than McCain.

Land line soft-voters are 8% soft for Obama [base =46%...so a share of that base is about 18% soft]. McCain goes from a base of 41% likely+leaners to 30% "strong" ~ a reduction of 11% [base = 41%...so a % loss of the base is 26%].

Landline users seem to be more decisive overall, and Obama supporters, though fewer, are more likely not to change their views.

So the irony here is that McCain has a chance to influence the cell phone only voters and change their position...while the landline voters are more firm in their views. Yet at the same time Obama has overwhelming support within the cell phone only class.

One more interesting point. Landline users with a cell-phone are almost identical in response to landline only users. It's those that are utterly independent of landlines that demonstrate significant differences.

Interesting that Obama has already started building a data base of cell-phone users which he can text message with registration and voting information.

____________________

Mark Lindeman:

@Michael: OK, thanks, close enough. You have a few more registered A-A voters and a bit fewer who didn't vote, and it looks pretty plausible. Yes, I imagine many of those registrations were deadwood.

____________________

distantobserver:

I believe Plouffe overreached a bit. The "Daily" is a valuable source, of course, but one that has to be handled with extreme care. Most of the change it seems to show is mere noise.

In a way the "Daily" is "the worst thing that happened to journalism" - but we shouldn't blame the daily but the silly interpreters. They see some change (most probably it's noise), and they immediately speculate about the political reasons for the change. Did Obama's statement do damage? Does the McCain ad work?

Even with a generated random walk they'd surely find some event that seems to explain the "shift"; so they'll always find a plausible explanation, thus disfiguring reality claiming to be "just interpreting the numbers"...

One final remark about Frank Newport's own role in this. Every day I read Gallup's own commenting of the "Daily", and while they always mention the possibility of changes being noise, more often than not they participate in the kind of distortion I mentioned.

I think this encourages pundits to make the same mistake - only more efficiently...

____________________

Gary Kilbride:

Plouffe or anyone else is kidding themselves of they think national polls don't matter in a general election, particularly the averaging of national polls. The nation is basically 21% liberal, 33% conservative. And not coincidentally that's also the makeup of the swing states. Many of the most vital ones tilt a point or so in either direction of that 21/33 split. It's rank foolishness to believe the same philosophical breakdown will react markedly different simply because the boundaries are varied.

The Obama campaign isn't magical and isn't inventing anything. It is subject to preference of the American people. If that dips or surges nationwide, it will apply state to state also, regardless of saturation coverage or what David Plouffe wants to believe.

I seem to remember the same crappiola in 2004. I warned on many sites that Ohio's vote would mirror the national margin, within one point either way. Sure enough, Kerry lost nationally by 2.5 points and that was sufficient to forfeit Ohio.

____________________

I'd have more respect for Gallup's polling if they ALSO polled and reported Electoral College status. Imagine if we had daily tracking polls in the 18 battleground states.

As it is, Gallup's daily average measures the totality of the hurricane, but doesn't tell me the only thing I care about: is MY house ok?

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR