Pollster.com

Articles and Analysis

 

Dispatches: Greenberg's Response to Franklin (Part 2)

Topics: Dispatches from the War Room , Pollsters , Stan Greenberg

This guest pollster contribution from Stan Greenberg is part of Pollster.com's week-long series on his new book, Dispatches from the War Room. Greenberg is chairman and CEO of Greenberg Quinlan Rosner.

Charles Franklin rightly begins his comments by putting up my quote on page 58 that "the endgame in presidential campaigns brings out all sorts of irrationalities, starting with the media polls. Many are criminally bad." One of the problems in writing a book and a memoir is living with your words and thoughts, particularly when as unnuanced as those.

Dispatches.jpg

In retrospect, I might have been more nuanced. First, I made the comment in the context of the Clinton presidential campaign when the statement was clearly true, as described in the book. Second, it reflects my experience during the final weeks in campaigns in Britain and Israel and in Latin America, even very recently. But because of sites like Pollster.com, there is more transparency and exposure of shoddy methods, and despite strong budget pressures, the national media organizations in the US produced very credible polling programs in this last election. But as recently as 2004, there were stark examples of volatile polls without political weighting conducted by Gallup and aired on CNN, along with commentary on how fickle were the voters. The challenge will be what happens with media polls, as there is more upheaval in the industry and need for more costly multi-modal methodologies and greater use of IVR.

This is a very different matter when one goes down to the state and congressional level and when you are in lower turnout elections and primaries. The media polls, as well as polls conducted by universities and institutes, are often out of line with the campaign surveys, as they are less likely to screen or filter for likely voters, factor-in historic turnout patterns and consider use of exit polls, as well as CPS. That one in four state polls in 2008 were conducted one day suggests we are dealing with a genuine issue.

I Amen, Franklin's Amen. The biggest problem is the reporting, not the polls themselves. It is the "outlier" poll -- not the boring average that gets headlines. But it is even worse in the war rooms I'm writing about that are poised to explode in the closing week of the campaign. It is the errant poll, not the average, that sets off the sparks in the war room and gets the attention of the candidate.

 

Comments
moderate2008:

It has been a great week here reading and discussing great issues with Mr. Greenberg and other contributors. Mark, congratulations on a great job in hosting this forum.
Mr. Greenberg states that, "the media polls, as well as polls conducted by universities and institutes, are often out of line with the campaign surveys, as they are less likely to screen or filter for likely voters, factor-in historic turnout patterns and consider use of exit polls, as well as CPS."
Here Mr. Greenberg's implication seems to be that use of the exit polls and other historical turnout patterns, along with the CPS data is what makes the campaign polls better than the media polls.
First of all, I would agree with Mr. Greenberg, only if it were true that most of the public/media polls did not use such information to come up with their likely voter scenarios. In fact, most of the public polls, at least the ones I know of, do screen for likely voters by factoring-in historic turnout patterns and do "consider" exit polls, as well as CPS in establishing their likely voter screens. Why wouldn't they? Why would they willfully choose to be reckless -- using an unlikely voter screen?
Second, using exit polls with CPS data is just another highly imperfect tool - like any other historical data - to intelligently and reasonably "guess" what a likely voter turnout will look like. In the end a 4 year old exit poll is exactly what it is supposed to be - a "poll" - with a MOE built in. Using these tools for establishing likely voter screen gives a particular pollster - campaign or media - as much advantage over another pollster, as a Wall Street investor gets over another investor looking at the same historical P/E charts. None. As Charles Franklin put it in his earlier post, polling techniques and methodology after all are "open source."
To another of Mr. Greenberg's point being "out of line" with campaign surveys - as he puts it - is not the same as being "wrong". Campaign pollsters should stop treating and touting their own internal polling as benchmarks against the public/media polls without releasing information/results of these polls. It seems kind of hypocritical to me - I hope I am not coming out too strong here - for the campaign pollsters to criticize the public polls without ever letting out their methods for scrutiny in the public domain and letting these methods to be compared against those of the public polls they take issue with.
I am not saying campaign pollsters should release their methodology and results - they have a right to keep it a secret and protect this information for their clients - I am just saying that until they are ready to compare their own techniques and methods, in a public way, with the media polls out there, they should refrain from the temptation of making unnuanced criticisms of these polls.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR