February 22, 2009 - February 28, 2009


46 "Outliers"

Topics: Outliers Feature

Kaiser's Drew Altman identifies a key health reform polling number to watch; Ezra Klein, Kevin Drum and Andrew Sullivan ponder it further.

David Weigel takes a long and skeptical look at Rasmussen Reports, joined by Josh Marshall and Matt Yglesias.

Gary Langer says Barack Obama's initial job ratings are strong, average for a new president, and as partisan as other recent new presidents.

Frank Newport finds that forty-somethings worry more about money.

Jennifer Agiesta looks at support for DC voting rights.

David Hill offers Obama advice and explains how pollsters evaluate political viability.

Tom Jensen wonders why IVR polls seem to produce different job approval numbers.

Wilson Research assesses the political environment.

John Sides says Stan Greenberg was right; finds "nothing particularly useful" in Gallup speech reaction poll.

Stan Greenberg booked on this Sunday's This Week with George Stephanopolous.

Paul Bedard ponders whether Frank Luntz is a uniter or a traitor.

Frank Luntz wants your thoughts.

Michael Link and colleagues explain "address based sampling" as means of reaching cell phone only households.

Congratulations to occasional poll parser and proud new papa Chris Cillizza.

The headline? A clue in this link.

US: 2012 Reps (CNN-2/18-19)

2/18-19/09; 429 Republicans, 4.5% margin of error
Mode: Live Telephone Interviews


'12 Presidential Republican Primary
Gov. Sarah Palin 29%, Mike Huckabee 26%, Mitt Romney 21%, Gov. Bobby Jindal 9%

Perspective Check: In February of 2005, a CNN / USA Today / Gallup survey of 383 Democrats found Sen. Hillary Clinton leading Sen. John Kerry (40% to 25%) in a national primary contest with John Edwards trailing at 18%.


Favorable/Unfavorable Views of Rush Limbaugh

Much discussion of the influence of Rush Limbaugh in the conservative movement and the Republican party these days. This week a Public Policy Polling (PPP) poll discussed demographic differences, and there has been some buzz about the gender gap found in that poll. But let's put this in some perspective.

Polling on Limbaugh began in March 1993, and I've now found 16 national polls asking favorable or unfavorable views of him, with PPP the latest.  Gallup and Democracy Corps have also asked this question within the last 4 months. 

Point number 1 is obvious. Rush burst onto the scene in the early 1990s, and immediate enjoyed a base of favorable supporters, amounting to 25-30% of the public. Initially there was substantial lack of awareness of him as well, as we'd expect with new media figures. Those unaware or unable to rate him were over 40% initially. But as his visibility exploded, this lack of awareness shrank sharply to about 20%, where it has remained ever since 1995.  

But growth of awareness was not accompanied by a gain in favorable audience. Instead, favorable response has remained virtually unchanged since the early 1990s. Favorability has fluctuated between 25 and 30%. What has grown is unfavorable evaluations--- from 30% before awareness expanded to a stable 50% unfavorable rating that has also been largely stable since 1995. 

So the first point of perspective is that Rush's base has remained stable and consistent from the beginning. He has not gained any share of favorable evaluations since 1993. That strong base of listeners has been hugely valuable to him but his influence does not rest on gaining widespread admirers. Rather it rests on devoted listeners who have stayed with him for 15+ years, which is a remarkable achievement.  It is this ability to speak to the base (CPAC this weekend for example, as I'm hearing him say as I write this). And I assume that's how he would like it. Broad inclusiveness is not the strategy.

Second point of perspective: The PPP survey is way out of line with 16 years of polling on this. PPP finds only 10% unable to rate Rush, while other polls remain around 20% unaware. Gallup in January found 11% not heard plus 16% unable to rate. Democracy Corps in November found 13% not heard and another 13% neutral.  So PPP seems to have tapped a population more willing to rate or more aware than other national polling. This is an IVR poll with the very low response rate such polls generally have. 

As a result, while discussion of the demographic breaks in the PPP survey are interesting, the levels of favorability reported among both men and women is substantially greater than  any other poll taken over the past 16 years. PPP found 46% favorable. The highest since 2000 is 34% by Gallup in 2003, and the three polls since then have been 26%, 23% and 28%. 

Bottom line, PPP has substantially overstated the level of favorability Rush enjoys among the public. His popularity and influence don't rest on the size of his supporters as a percent of the population but rather their loyalty and their share of radio listeners, a different population than adults nationally. 

NC: Obama, Economy (Elon-2/22-26)

Elon University
2/22-26/09; 758 adults, 3.6% margin of error
Mode: Live Telephone Interviews

North Carolina

Obama Job Approval
Approve 59%, Disapprove 25%

Do you [support or oppose] the federal government stimulus package?

52% Support
39% Oppose

Do you think the funding that the federal government provides through the stimulus package will have a [positive or negative] effect on the economy?

54% Positive
30% Negative
7% Neither

US: Obama's Speech (Gallup-2/25)

Gallup Poll
2/25/09; 1,025 adults, 3% margin of error
Mode: Live Telephone Interviews


"President Barack Obama's address to Congress Tuesday night appears to have bolstered confidence among many Americans. Four in 10 (41%) say they are now more confident in his plans to improve the economy, including 57% of those who watched or listened to the speech live.

In a one-night Gallup Poll conducted Wednesday night, Americans overall were about evenly split between saying Obama's speech made them more confident and saying it had no effect on their opinion. But those who reported watching or listening to the speech live were far more likely to say it made them more confident, out numbering by a 2-to-1 margin those who said it had no effect. Fewer than 2 in 10 Americans in either group said the speech made them less confident."


US: Economy (ABCPost-2/12-22)

ABC News / Washington Post
2/12-22/09; 1,001 adults, 3% margin of error
Mode: Live Telephone Interviews


Just your best guess - do you think the economy (is in a normal downturn that will correct itself before too long), or do you think the economy (has moved into a serious long-term decline)?

    42% Normal downturn
    56% Serious decline

How much longer do you think the country's current economic recession will last - a few more months, up to six months, up to a year, up to two years, or longer than that?

    28% A year or less
    36% Up to two years
    34% Longer

(story, results)

US: Health Care (Kaiser-2/3-12)

Kaiser Health Tracking Poll
2/3-12/09; 1,204 adults, 3% margin of error
Mode: Live Telephone Interviews


"In the face of the country's current economic challenges, the public's support for health reform remains strong and their trust in President Obama to do the right thing in health care reform is high.

Slightly more than half (53%) of Americans say their household cut back on health care due to cost concerns in the past 12 months. The most common actions reported are relying on home remedies and over-the-counter drugs rather than visiting a doctor (35%) or skipping dental care (34%). Roughly one in four report putting off health care they needed (27%), one in five say they have not filled a prescription (21%), and one in six (15%) say they cut pills in half or skipped doses to make their prescription last longer."


State of the Country
19% Satisfied, 73% Dissatisfied (chart)


Gould: Greenberg versus Penn, Continued

Topics: Dispatches from the War Room , Mark Penn , Philip Gould , Pollsters , Stan Greenberg

[This Guest Pollster contribution comes from Philip Gould, who served as a polling and strategy adviser to the British Labour Party for general elections held from 1987 until 2005.

Editor's note: Gould was a central figure in the dispute between pollsters Stan Greenberg and Mark Penn that we have covered this week, as he was responsible for managing the services that each provided to the Labour Party. He submitted his comments to Pollster.com in an effort to help clarify and resolve some of the issues raised here this week.

Since I emphasized the question of whether Penn delivered complete marginals and cross-tabulations, I want to promote the following paragraphs that come toward then end of Gould's memo:

After a poll Stan normally presented a filled in questionnaire, a full banner book containing complete cross tabs.

Mark had a different approach. Following a poll he quickly made available a full and extensive polling report. This went immediate to the whole campaign. This was not an inconsiderable document. I have one in front of me now: it is 18 pages long; it contains historic voting and favourability data; it closely examines 12 targeting groups ranging from rural lower class Conservatives to union households; it uses seven different batteries to examine campaign issues. It analyses responses to the news and key policy areas. And of course it contains numerous message batteries: in all well over 100 questions were asked and recorded. All of these were analysed by voting preferences, and sometimes by demographic categories.

These reports were extensive and useful documents, far in excess of a normal filled in campaign questionnaire. They did not constitute a full banner book and did not contain 'full marginal's' in the manner favoured by Stan Greenberg, but what Penn did supply was both exhaustive and useful, and certainly met the regular needs of the campaign. As one senior campaign official with responsibility for polling in 2005 has said: 'Mark Penn 'could quite fairly argue that the memos were intended for an audience that had no time or interest in delving into every corner of the data. I don't think that in any way illegitimises the findings or his advice'. On a personal note Mark Penn invariably supplied any additional cross tab or targeting data that I required, and I presume the same is true of others. Two pollsters, two approaches.

Gould's piece covers far more ground than this narrow excerpt.  It is well worth reading in full. 

-- Mark Blumenthal]

I am aware that intercession in the Greenberg/Penn polling war can precipitate what has probably never happened before: uniting Stan and Mark in the face of a common enemy (i.e. me). But with all the risks it entails I will press on. From the start I must declare an interest- I suspect I am one of the very few people around who can claim that they like and respect both Greenberg and Penn (I can already feel them starting to unite against me!). I worked with Stan for well over ten years and believe him to be an outstanding pollster and strategist. I worked with Mark for a much shorter time, and came to greatly appreciate his skills too, different from Stan's certainly, but considerable for all that. It is in that spirit that I write this piece.

There are so many issues here, of methodology, strategy, personality and of course memory that getting to the truth of what actually happened in the UK election campaign of 2005 is probably impossible, but I will try at least to clear away some of the fog. Not by focusing on the smaller, although I accept crucial disagreements between the two pollsters, but by trying to paint a bigger picture, and using where possible contemporary sources, notes written at the time, my rather sketchy diary, and in particular a lecture I made to the LSE on the campaign in 2006 which pretty accurately sums up what I believe about the campaign.

[Continue reading after the jump]

Continue reading "Gould: Greenberg versus Penn, Continued"

Speech Reactions

Topics: CBS , CNN , Knowledge Networks , Speech Reaction , State of the Union

Some initial survey and focus group reactions to tonight's "non State of the Union" speech by President Obama.

CBS News conducted a nationally representative online poll of 500 Americans that watched the speech.** "Seventy-nine percent of speech watchers approve of President Obama’s plans for dealing with the economic crisis. Before the speech, 62 percent approved." More results here; they promise a complete report later tonight.

CNN conducted a survey 484 adults who watched Obama's speech. "Sixty-eight percent of speech-watchers questioned in a CNN/Opinion Research Corporation survey had a very positive reaction, with 24 percent indicating that they had a somewhat positive response and 8 percent indicating that they had a negative reaction."

Democratic pollster Stan Greenberg's Democracy Corps conducted a 50-person "dial group" (a non-random sample focus group in which participants register their reactions throughout the speech). In a post-speech conference call, he described the speech as successful for Obama, with much less polarization separating the responses of Democratic and Republican participants than is typical. "I've never seen anything like it. Republicans never went below 50 [on their dial ratings]." Democracy Corps has posted a report (including video of the dial test) and tables (PDF) will post a complete report either later tonight or tomorrow.

Republican pollster Frank Luntz conducted a 27-person focus group for Fox News ("half voted for Obama, half voted for McCain") that "divided along party lines." Fox has posted video and a transcript of their on-air summary. Huffington Post's Sam Stein has more.

MSBNC conducted a "dial group" of 32 Obama and McCain supporters from three Pennsylvania Counties. MSNBC's Tamron Hall reports via Twitter that "both groups gave the Pres extremely high marks. Approval number went up with dems and republicans." The Washington Post's Jonathan Capehart has more.

Update - NBC has posted video of Tamron Hall's summary of their focus group:

Also, ABC News polling director Gary Langer advises us to take all of this with a grain of salt:

We at ABC quit producing instant speech-reaction polls several years ago. Good sampling's a bear in this kind of thing, but there are two equally basic problems: Speech watchers tend to be favorably inclined to the speechifier in the first place (those who can't stand him are unlikely to watch); and speeches are crowd-pleasing (even platitudinous) by design (e.g., let's cure cancer).

While viewers may get caught up in the moment, a single speech in and of itself is very highly unlikely to change any fundamental attitudes. Events on the ground do that. People do listen to what our leaders say - but above all, we watch what they do.

**CBS News samples speech watchers using the nationally representative Knowledge Networks internet panel. The original version of this post put quotation marks around the words nationally representative, which struck some readers as expressing editorial skepticism about their methodology. Apologies for that, as I intended no such meaning. That said, as I explained back in August, there are reasons to be a bit more cautious about interpreting results from a panel survey.

Omero: (Obama's) Policies Before Bipartisanship

Topics: Approval Ratings , Barack Obama

Today's NYT/CBS poll has made news by suggesting bipartisanship may take a back seat to policies.  At least, if those policies are those of President Obama.  A majority (56%) want to see Obama work on "the policies he promised he would during the campaign."  Fewer said he should work in a bipartisan way with Congressional Republicans.


By contrast, Republicans are strongly urged to work in a bipartisan way with President Obama.  Eight in ten (79%) say they should work with their Democratic colleagues and the President.  Only 17% felt they should "stick to Republican policies."


This suggests voters prefer policies over process.  If voters agree with the policies, the specifics of the process may matter less.  And with Obama at 63% approval rating, while Republicans in Congress suffer from a 56% disapproval rating, voters seem to be making some early decisions about who has their best interests at heart.  Something to remember in the debate after Obama's speech tonight before Congress.

A Little Perspective on Early Approval

There have been a number of selective comparisons of President Obama's approval ratings one month into his presidency. Some stress change since earlier readings, some focus on comparison to a previous president, though seldom more than one. And some find stable high approval while others note a first reading below 60 (in their poll, not in all polling.) So let's bring some perspective to this by simply looking at all the data.

The chart above shows all newly elected presidents since 1952. The historical data are from the Gallup Poll, the only organization with such a long term history. And for Obama we have all the polls conducted so far by anyone, a total of 80 counting daily trackers by Gallup and Rasmussen. 

The previous presidents didn't "enjoy" such intensive polling, so one challenge is choosing which early polls to use in a comparison. While each previous president got at least one poll in their first thirty days, others waited until nearly 60 days for the second. This makes comparison to previous presidents "at the 30 day mark" a bit of an approximation.

President Obama has the opposite problem. With 80 polls so far the issue is which to choose. ABC/WP at 68%? CNN at 67%? CBS/NYT at 63% USAToday/Gallup at 62%, Gallup Daily at 59% ("first time below 60"), Rasmussen Daily at 60%, Greenberg/Democracy Corps at 58%? 

How about changes? CNN is a whopping 9 point drop, from 76 (an outlier) to 67. But CBS/NYT finds a 1 point rise from 62 to 63. USAToday/Gallup down 1 point, 63 to 62. Gallup Daily down 9 from the first post-inaugural 68 to 59. Rasmussen down just 1 point, 61 to 60 from first post-inaugural to now (but their overall trend is down). 

So let's try a little perspective. 

First, across all 80 polls, there has been a small downward trend in approval of Obama, from 63.7% to 60.9% based on my local trend estimate (the solid dark blue line in the figure.) 

There has also been a lot of scatter in the polling. The open blue circles in the figure show all 80 polls. We've seen lows of 52 and 54 and a high of 76, but the substantial majority of approval readings are between 60% and 67%.

That range of readings puts Obama not at the high end nor at the low end of post-war presidents.  Kennedy, Eisenhower and Carter were consistently higher early on. Nixon started slightly lower, but moved up into Obama's range. Both Bushes started a little lower but then moved up to 60% or better briefly then back to the 50s. Reagan and Clinton were consistently lower and didn't challenge the 60% range for some while. So the clear evidence is that Obama enjoys good but not extraordinary approval ratings for the first month of his presidency. 

There is some interesting variation across the polls. The chart below shows that the Gallup and Rasmussen daily tracking polls are both trending down on parallel trajectories. But the non-tracking polls, what we used to think of as the "normal" national polls, are flat or even a bit up (the early outliers pull the line down at the beginning, without them it is basically flat). Put it all together, and you get the blue line trend, with that modest 63.7% to 60.9% decline.
It is an interesting puzzle why the two daily trackers see a downward trend while the other national polls don't. Part of the answer is the role of house effects and the small number of non-daily polls. That the most recent polls are towards the middle or upper range of all polls makes the black "other polls" trend run flat and high. The advantage of the daily polls is they use the same methodology every day, so their trends can't be flukes of mixing different methods. But between the two daily polls there is a house difference of about 4 percentage points, so clearly there are methods artifacts here as well.  The "all polls" trend estimate splits these differences pretty reasonably, but the variation you see here means there are lots of opportunities to cherry pick a poll you like, whichever side you like. 

Finally, let's remember all this doesn't mean much of anything for the long-run success of a presidency. President Clinton started low but ended with the best final approval poll of all these post-war presidents. President Carter started high but ended low. The chart below tells the tale:
Statistically, there is no correlation between standing at the second poll (roughly 30 days) and final approval poll. It doesn't matter if we use any of the polls taken in the first 90 days. There is simply no relationship between early performance and the public's judgment once the presidency is at its end.  So let's be a bit slow to place too much emphasis on these early polls. At this point, we are 2% of the way through President Obama's term. Plenty of time left to learn if he succeeds or fails.

Smith Reports on the Greenberg-Penn Feud

Topics: Dispatches from the War Room , Mark Penn , Stan Greenberg

Over at Politico, Ben Smith picks up the story of the Greenberg-Penn feud:

Greenberg’s new book ups the stakes. He argues that Penn played dirty inside the Blair campaign, tweaking the questions and introductions in his polls to produce results that favored his “mindless, fixed theory.” Through “biased wording,” Greenberg writes, “the tests were rigged.”

Penn fired back Monday on a polling website and in an interview with Politico, calling Greenberg’s charges of rigging “ludicrous,” his strategic theories “unsubstantiated,” and his attack the product of “sour grapes.”

The spat between the two men comes, perhaps not coincidentally, as Greenberg is selling his new book, Dispatches from the War Room. Their dispute offers a behind the scenes glimpse into the big-ego, big-bucks world of top-shelf political consulting, where franchises and reputations are made being the kingpin in a presidential campaign.

The small fraternity of top Democratic pollsters watched the fracas with amazement and amusement Monday, and sniped from the sidelines.

“This is ridiculous – this isn’t about polling and strategy,” said one, describing it as “a pissing match” that will, incidentally, “sell books.”

On a somewhat related and personal note, we will be back to other topics soon. I'm still depleted from a 24-hour viral bug that hit me yesterday, so apologies for less productivity than usual over the last two days.

NYC: 2009 Mayor (Quinnipiac-2/17-22)

Quinnipiac University
2/17-22/09; 984 registered voters, 3.1%
646 registered Democrats, 3.9%
Mode: Live Telephone Interviews

New York City

Job Approval
Mayor Bloomberg: 65% Approve, 29% Disapprove

Favorable / Unfavorable
Mayor Bloomberg (i): 62 / 32
Weiner (D): 32 / 12
Thompson (D): 22 / 5

'09 Mayor Democratic Primary
Weiner 32%, Thompson 22%

'09 Mayor General Election
Bloomberg 48%, Weiner 36%
Bloomberg 50%, Thompson 33%


NY: 2010 Sen, Gov (Siena-2/16-18)

Siena Research Institute
2/16-18/09; 622 registered voters, 3.9% margin of error
Mode: Live Telephone Interviews

New York State

Job Approval
Gov. Paterson: 28% Excellent/Good, 69% Fair/Poor

Favorable / Unfavorable
Pres. Obama (D): 74 / 18
Gov. Paterson (D): 40 / 47
Sen. Gillibrand (D): 34 / 20
Giuliani (R): 58 / 35
Cuomo (D): 69 / 18
Lazio (R): 23 / 23
King (R): 23 / 15

'10 Governor Democratic Primary
Cuomo 53%, Paterson 27%

'10 Governor General Election
Giuliani 51%, Paterson 36%
Paterson 46, Lazio 28
Cuomo 51%, Giuliani 38%
Cuomo 66%, Lazio 16%

'10 Senate General Election
Gillibrand 40%, King 27%


A Tuppence Worth

Topics: Dispatches from the War Room , Mark Penn , Pollsters , Stan Greenberg

A reader, a European politico who "witnessed the working of the Labour war rooms in both 2001 and 2005," emails to share his "tuppence worth" on the Greenberg-Penn exchange:

In terms of the atmosphere of the campaigns - 2001 was one large open-plan office where even [then Chancellor of the Exchequer Gordon] Brown sat. It was an upbeat room where there was a constant interchange between people. Greenberg was deeply embedded within everything. 'Loops' were large and delivered a brilliantly effective campaign.

2005 was factionalised, with people working in separate (large) rooms and with lousy communication. It was a very unhappy place. Penn's operation had a much larger physical presence than Greenberg's had had - reflecting a tendency to be self-contained in analysis versus Greenberg's more collegial approach (in 2001 I overheard him go to some Brits with "these are just in what do you think?"). Alistair Campbell is credited with going to Blair and saying 'enough' and things got back on track.

I have never worked with Greenberg, and he wouldn't know me, but I have observed his work closely in a couple countries and his approach is clearly to be a partner of the client - essential to recognising the limits of US experience when applied to international campaigns. This is why his is the only US firm which has had a long-term presence in Europe. Penn is building his business in Europe, but we'll have to wait a few years before we see if it's sustainable.

Dispatches: Penn's Rejoinder to Greenberg

Topics: Dispatches from the War Room , Mark Penn , Pollsters , Stan Greenberg

Mark Penn has emailed a second response to Greenberg, which I have reproduced in full below.

One aspect of this back-and-forth is still not quite resolved. In his book, Greenberg writes that "Penn's firm provided none of the information normally delivered by a professional research organization." In his rejoinder posted earlier today, Greenberg expands on that claim:

Pollsters as a rule share the results for all their questions and hypotheses, even the ones that didn't pan out. In the Blair campaign, Penn provided a memo with large tables including only the questions he wanted to report; he did not provide a standard book of demographic cross-tabulations. Read Penn's words carefully, "The campaign received all of the agendas, marginals, as requested without reservation." In short, he provided breakouts only when asked, in effect keeping his own client and campaign team "out of the loop."

Penn's latest response does not directly answer Greenberg's charge:

The questions in the polls went through a team vetting process and it was up to Labour not me to determine what Stan got and what Stan did not get. I have plenty of letters of transmittal of questionnaires, marginals and crosstabs to the party - they chose what Stan did and did not get - and there were very sensitive questions for small group use only that they decided not to give him.

So Penn's firm transmitted "plenty" of questionnaires, marginals and crosstabs, but when and how often? If these documents were only shared on request, and then only after strategy meetings or conference calls in which Penn presented results (as was reportedly the case in Hillary Clinton's campaign), then Greenberg has a point. The standard procedure of every pollster I've worked with is to share these documents with the campaign as soon as they are ready -- by email or (in the good old days) by fax -- so that campaign decision makers can examine the data and be in a position to question the pollster's strategic recommendations. Perhaps this is a question that Phillip Gould can help resolve.

Penn's latest response follows after the jump.

Continue reading "Dispatches: Penn's Rejoinder to Greenberg"

US: National Survey (CBSTimes-2/18-22)

CBS News / New York Times
2/18-22/09; 1,112 adults, 3% margin of error
Mode: Live Telephone Interviews


Obama Job Approval
63% Approve, 22% Disapprove
Economy: 57% Approve, 32% Disapprove

Congressional Job Approval
26% Approve, 63% Disapprove

National Economy
8% Good, 91% Bad
8% Getting Better, 51% Getting Worse, 40% Staying the Same

From what you know, what impact will the stimulus package have on the current economic impact?

53% Better
13% Worse
24% No impact

(CBS story, Iraq, Michelle Obama, Economy, Times story, results)

US: National Survey (ABCPost-2/12-22)

ABC News / Washington Post
2/12-22/09; 1,001 adults, 3% margin of error
Mode: Live Telephone Interviews

Obama Job Aproval
68% Approve, 25% Disapprove
Economy: 60% Approve, 34% Disapprove

Direction of Country
31% Right Direction, 67% Wrong Track

As you may know, the federal government will spend about 800 billion dollars on tax cuts, construction projects and aid to states and individuals to try to stimulate the economy. Do you support or oppose this plan?

    64% Support
    30% Oppose

Do you think the economic stimulus plan will help or hurt your own personal financial situation?

    46% Help
    35% Hurt

(ABC story, results; Post story, results)

Dispatches: Greenberg's Rejoinder to Penn

Topics: Dispatches from the War Room , Mark Penn , Pollsters , Stan Greenberg

This guest pollster contribution from Stan Greenberg is part of Pollster.com's week-long series on his new book, Dispatches from the War Room and responds to comments from Mark Penn in Mark Blumenthal's post earlier today.  Greenberg is chairman and CEO of Greenberg Quinlan Rosner.

To avoid this discussion descending into an ugly mud-wrestling match between two squabbling pollsters, I will only take up issues where the "facts" are indisputable and where we learn something about Tony Blair and political leadership and about differing approaches to polling and strategy.

What this exchange reveals even more clearly than the book itself are the limits of building a strategy from a coterie of target groups, rather than from the leader's vision or party's mission for the times. It underscores the need for frankness about what is holding voters back and the need to challenge leaders with blunt truths. It underscores the need for transparency and methodological rigor.

Penn's basic argument is straightforward. He took over the campaign's polling in July 2004 about nine months before the election when Blair was at a low point, working under Philip Gould, Blair's long time advisor for research and media. Greenberg was pushed out and was in no position to judge the character of Penn's work, as he was "not in the loop." Seems straightforward enough.

When I first learned in December of Penn's involvement and in January of our dividing the polling, I was convinced that Gould had played just such a role and I wrote about it. I was wrong. Philip was hurt by the accusation that he had concealed Penn's involvement and wrote me with detailed diary entrees that show he only learned of it in September and resisted Penn's involvement until the end of the year, when he decided to "make the best of it."

Penn's premature rush to anoint himself as Blair's pollster obscures Blair's effort to examine competing solutions to the problems he faced. In May, Blair had reached a low point in the polls, dragged down by Iraq, the "hyping" of pre-war intelligence and Abu Ghraib. He was very despondent, seriously considering not running again and consulted widely, including with President Clinton and Senator Clinton who urged him to run and to use Penn.

Penn offered his own path back for Blair, aided by huge surveys and "clustering work" that coughed up "school gate mums" as a key target. Because Labour got its highest marks on the economy, his message started there, but Penn's emphasis was on policies that appeal to the groups that can grow Blair's coalition. Penn's imprint was immediately evident in Blair's September conference speech when he spoke of the stresses of the need for "more choice for mums at home and at work." Blair's policy offer was grounded in this clustering and coalition building.

At the very same time, we were commissioned by Phillip to do a special research project and I reported in July with a very different approach to the problem - centered on New Labour's central mission. For the first time in a long time, respondents shifted to Labour on hearing of Blair's commitment to "a better life for hardworking families," though only when Blair expressed his own frustration with the state of public service reform and offered some learning by showing independence from Bush on climate change. Iraq was the elephant in the room. Finding a way to acknowledge it, even indirectly, allowed people to come back to Blair's project.

In the September party conference speech, Blair was eloquent about "hardworking families," but just could not get himself to be reflective on Iraq - perhaps with Penn's support. That was the learning voters needed if they were to come back.

I respect Blair for rejecting my advice and deciding to go with Penn who did not push him to address the Iraq question and who offered a way to make electoral gains. The mistake was not firing me and leaving both of us in the campaign.

In fact, I have all of Penn's memos - about a two-inch pile on my desk at the moment, available for inspection by Mr. Blumenthal. Philip's note to me confirms he shared all of them during the course of the campaign, as did many of my friends "in the loop."

The whole concept of "in the loop" betrays a lack of transparency and openness in Penn's approach to campaigns - painfully evident in the Blair campaign, perhaps a precursor to Hillary Clinton's presidential run two years later.

Pollsters as a rule share the results for all their questions and hypotheses, even the ones that didn't pan out. In the Blair campaign, Penn provided a memo with large tables including only the questions he wanted to report; he did not provide a standard book of demographic cross-tabulations. Read Penn's words carefully, "The campaign received all of the agendas, marginals, as requested without reservation." In short, he provided breakouts only when asked, in effect keeping his own client and campaign team "out of the loop."

The surveys were methodologically sloppy and included biased tests, though it is important to underscore here that Philip Gould came to value Penn's research and rejects my characterization of it in the book.


1) Penn failed to incorporate professional learning from Britain. Penn national polling - not some errant tracking program - showed Labour with landslide leads of 8 or 9 points for the entire six weeks prior to the election being called. Penn discovered just 27 days before the election what every pollster in Britain has knows: you have to weight to offset the "shy Tories" - Conservatives reluctant to be interviewed. In an instant, the Tories gained 6 points in Penn's polls.

2) Penn's fixed targeting let real targets slip away. With Penn focused on "mums," the campaign regularly rolled out initiatives on breast cancer screening and childhood obesity. But voters in the key marginal seats were older and among those most likely to return to Labour, two-thirds had no children at home and found this campaign irrelevant.

3) Penn exaggerated the reliability of findings. Penn conducted a valuable weekly open-ended Internet panel of undecided voters. When the sample dropped to 100, so did the reporting of sample size that produced a testy email exchange that restored it. Still, Penn reported this as a "Survey of Undecided Swing Voters" and reported the full percentage results over 18 pages, including results for men and women, with about 50 cases each.

4) Penn created biased tests. Two weeks before the election, Penn declared that "our policy approach remains stronger than the Tories," but the Labour statement was more than twice as long, with more rhetorical flourishes and covering a much broader range of policies with greater specificity (which I'm happy to share). Even with this biased test, the Conservative's statement ran 6 points ahead of its vote. An unbiased test might have revealed potential Tory gains.

To inform the decision of whether to close positively or negatively, Penn constructed a sensible experiment where half the respondents were read positive statements about Labour's progress and half read attacks on the Conservatives' record and plans, and then respondents were asked to vote again. But this was not meant to be a fair test. The negative statements were 50 percent longer by word count and helped foreclose an uplifting close.

Penn describes the 2005 third-term as "historic" but in the campaign everyone was disappointed with the result, what the media called Labour's "drastically reduced majority," produced by a disengaged electorate and historically low turnout. Many factors contributed to the result, but among them were Penn's research, not to mention having two polling teams with different theories on how to win.

Dispatches: Greenberg vs. Penn

Topics: Dispatches from the War Room , Mark Penn , Pollsters , Stan Greenberg

Of the many stories in Democratic pollster Stan Greenberg's new book, Dispatches from the War Room, the most newsworthy may be his slashing condemnation of Mark Penn, the pollster that displaced him within the inner circles of both President Bill Clinton and British Prime Minister Tony Blair.

Greenberg, reacting to what he saw of Penn's polling when both worked on Blair's 2005 re-election, describes Penn's methods as "errant," his tests of messages as "biased" and "rigged" and his documentation lacking "transparency" and "the information normally delivered by a professional research organization."

Penn, in response to my query, defends his surveys for the Labor Party as "extremely accurate," says Greenberg was excluded from information because he was "not in the loop," and describes Greenberg's attacks as inaccurate and "unsubstantiated."

This exchange brings out into the open a particular critique of Penn that until now came mostly from speculation or the reports of anonymous sources. Some aspects of their stories are in conflict, if nothing else the undisputed facts illustrate the contrast between the Carville/Greenberg "War Room" model of campaigns and the style of consulting that Penn practices.

[This post is part of Pollster.com's series on Dispatches from the War Room].

* * * *

For those just joining this conversation, Greenberg's new book chronicles his work for five national leaders, including Clinton and Blair. The story starts with the 1992 campaign that put Clinton in the White House and made the "War Room" famous. After the 1994 elections that swept Republicans into control of both the House and Senate, however, Clinton started taking advice from Dick Morris and brought in pollster Mark Penn to replace Greenberg and conduct polling for the 1996 reelection campaign.

In the midst of the first Clinton term, Greenberg also went to work for Britain's Labour Party, helping to create another political "War Room" that helped Blair and Labour win landslide victories in 1997 and 2001. Following the terrorist attacks of September 11, 2001, Blair emerged as one of the closest allies of the U.S. and President George Bush in both the response to the attacks and the Iraq War. As Greenberg relates in the book, he personally opposed the Iraq war and felt uncomfortable with Blair's embrace of Bush and military action in Iraq. Nevertheless, he remained an advisor since he "did not question Blair's motivation" nor his commitment to the larger "political project" that "transcended this war."

Ultimately, however, the war took a toll on both Blair's popularity and his relationship with Greenberg. In this period, Greenberg recounts several examples of Blair rejecting his counsel as he struggled with how to win back public support and whether to seek reelection. In November 2004, after Blair decided to seek a third term, the Labour Party informed Greenberg that they had retained Mark Penn and that Blair wanted "the polling program split between the two of you."

Greenberg soon learned that during periods of Blair's doubts, "President Clinton and Hillary, too, called repeatedly to urge Blair to run again...They also pushed for Blair to use Mark Penn to help find a way out." Penn had conducted polls kept secret from Blair's inner circle, even from Phillip Gould, the polling and strategy advisor. Friends also told him that the plan had been to "fire" him, yet "somehow nobody did it." So he continued to conduct polls on behalf of Labour in marginal districts and, as such, remained present in the campaign,

watching it all from a dysfunctional new war room. The Penn people were grouped at one set of long tables and my people at another, a touch further away from the cluster of tables at the center. It was a daily humiliating experience, but I decided I had too much invested in "New Labour" to walk away.

Greenberg's criticism of Penn stems from what he observed during the period including, by his account, both the shift in message and the results of Penn's polling. He recounts an episode in March 2005 in which Penn's polls showed Labour "with landslide leads," while Greenberg's surveys in marginal districts were closer and his review of public polls showed Labour leading by only four percentage points. A memo from Greenberg to Blair about these numbers "led Philip [Gould] to erupt in the war room," shouting at Greenberg's staffer to "tell Stan these emails to Tony are wrong and very destructive." In the end, Labour won by just three percentage points and a significantly reduced majority.

After the election Greenberg wrote three memos to Blair and then Chancellor Gordon Brown that interpreted the results and "put on paper in an inescapable way my outrage at the polling and the banal strategy that followed from it that put Blair and Labour at risk." He describes the overall "research program" (which included work by Penn and by others) as "unprofessional and lacking in methodological rigor...biased, self-deluding and overly optimistic."

The memoranda, as recounted in the book, include these specific references to Penn's work:

Penn's errant polling methods had produced landslide predictions all along, giving way in the last week to a roller coaster: first Labour crashed, then it surged back. One could only divine the cause because Penn's firm provided none of the information normally delivered by a professional research organization," I wrote. The lack of transparency allowed findings to be "packaged in scientific surety that should never have been reported with anything but the greatest qualifications."

The biggest problem was that the campaign fixated on winning over some Americanized target group. Without any evidence, Penn touted that "soccer moms" were the key swing group in the 2004 U.S. elections. In Britain that group became "married mums." The problem was that three quarters of the "lapsed Labour voters" in the marginal seats did not have kids at home. Penn set the campaign on a course that missed most of its key targets and would not adapt. So, "during the campaign, we successfully raised Labour's support with women with children at home" but "that came at a price" among older voters and pensioners and men under forty-five who were not stirred by yet one more initiative on breast cancer screening and childhood obesity. "It was as if creating an American-style gender gap was something to emulate, when Democrats in the U.S. have had so much difficulty winning nationally." I concluded that this mindless, fixed theory had "consequences": "a reduced Labour vote share and a reduced parliamentary majority."

After some criticism of focus groups by the campaign (but not by Penn's firm), Greenberg concludes, again quoting from his own memorandum:

[T]o keep the campaign on its predetermined course, "the national survey's were riddled with questions whose biased wording seemed to get the reports to a preferred conclusion." In short, the tests were rigged.

What makes these accusations unique is that they are from an on-the-record, first person account. Similar accusations -- such as not not sharing  "filled in" questionnaires (also known as "marginals") and crosstabs or cherry picking results to support a favored position -- have bubbled up in the past, but Greenberg is the first I can recall willing to make such accusations out in the open.

I emailed Mark Penn for comment and he sent a response that characterizes Greenberg's charges "unfortunate" and "unsubstantiated," defends his firms' data and its contribution to Labour's 2005 campaign. "We did in fact change the course of the campaign, developed a new message, a new set of targets, were extremely accurate, and received extensive written and personal praise from the prime minister." He denies as "ludicrous" the accusation that his polls showed Labour winning by landslide margins: "We never predicted a landslide. Most of our polls showed Labour and Tories within 4-5 pts. It was Stan's predictions of a loss that proved inaccurate."

Penn's complete response appears after the jump, but the following paragraphs are most relevant:

First and perhaps most disturbing is the idea that he did not receive any standard market research information - of course HE didn't. The campaign received all of the agendas, marginals and crosstabs as requested without reservation. But Stan didn't because he was not in the loop as 1) many of the polls contained highly sensitive questions and 2) he was seen as highly adversarial looking only to undermine the team and their effort - which is rather borne out by his book. The memo he quotes in his book actually sealed his fate and was seen as highly unprofessional and mocking of an effort led by a hard-working team of smart professionals including Philip and Alistair Campbell. Blair specifically praised the professionalism, quality and creativity of the work.

Penn may have had a different memo in mind, but to be clear, the internal memos that Greenberg quotes in the passages above were written two weeks after the 2005 elections. Penn continues:

Again, Stan was excluded from the strategy sessions and all meetings with the prime minister and provided with only limited information at the specific instructions of the client. How could he have understood what was going on when wasn't there for the high-level discussions? This is rather well illustrated by his saying he learned of us in December and that Philip Gould did not know of our work. In fact, we started many months earlier in the summer when the most in-depth positioning and clustering work was done and Philip was at every presentation and supervised us. Of course he did not tell Stan that he was working with us long before or that he had all of the information on the polls for reasons that are quite obvious.

And on top of all this Stan has confused daily tracking in the last month that was done by another firm (not Penn and Schoen) that Philip had arranged to have shared with the campaign. This daily polling did tend to shift around and we did not have much information on it - it was not used for much of anything. We did not do daily tracking - we did the message and strategy polling, and we supplied the questions, the answers, the sample sizes, and the logic behind each and every conclusion or suggestion.

We worked almost a year on this campaign asking hundreds of questions of all types and nothing was "rigged." Philip had input to all of the questionnaires without reservation as did members of the team and he ran extensive independent almost daily focus groups alongside us.

There are clearly some facts in dispute in this exchange, and resolving the conflicting stories is beyond the scope of this post. Other issues, such as whether question wording is slanted or "rigged," will inevitably involve subjective judgments. I assume that he will have more to say in response, and we look forward to trying to dig deeper into this controversy.

Timing is important in all of this. If "the campaign received all of the agendas, marginals and crosstabs as requested without reservation," when exactly were they requested and received? The complaint I heard from within Hillary Clinton's campaign was that Penn only shared filled-in questionnaires with the campaign's senior staff after the meetings or conference calls at which results were presented and decisions made, and only shared cross-tabs reluctantly on the condition they be locked up in the campaign manager's office.

But if we set aside the conflicting aspects of the two pollsters' accounts, we can still see a clear contrast in the way they approach campaigns and a vivid illustration of the Carville/Greenberg "War Room" philosophy. As Greenberg writes, James Carville's vision was to "house...all the key campaign people [in a single room], the opposition researchers, policy people, and news monitors." Yes, as most of us know, all of it was "geared toward fast analysis and instant response," but speed alone was not what made the concept unique.

What really distinguished a Carville War Room was the absence of doors, the way information flowed freely among the staff that worked inside. Instead of limiting access to "sensitive" polling information to the half-dozen or so in the campaign's "inner circle," Carville opened access to everyone working in the War Room, including relatively junior staff. "The idea was that information was widely shared among the campaign people," Carville himself told me when when I spoke to him on Friday. Even though he routinely shared sensitive strategic information with hundred or so staffers in the Clinton campaign "we never had leaks" (though to be clear: Carville was not involved in the 2005 Labour campaign).

In Penn's conception, polling information should be shared only with a select few with the candidate's ear, those "in the loop." Carville and Greenberg's "war room thinking" trusts everyone in that room with the details of strategy and polling so they can be, as Greenberg puts it, "sensitive to all kinds of information, and anxious to jump on any sign of strategy going awry."

Trust your staff with the data that drives the strategy and they are more likely to understand it and react accordingly. Limit "the loop" to just a small handful in a presidential campaign and expect infighting, dysfunction and poor execution to follow.

[Past relationship disclosed: I worked in Stan Greenberg's company, then known as Greenberg-Lake: The Analysis Group, in 1990 and 1991].

UpdateGreenberg responds to Penn, Penn responds to Greenberg.

Greenberg's full passage from Dispatches on this subject and Penn's response follow after the jump.

Continue reading "Dispatches: Greenberg vs. Penn"

Oscar Roundup

Edison Research conducted their first ever exit poll of 915 attendees of the AMC Best Picture Showcase "movie marathon" yesterday. It found Slumdog Millionaire leading Milk (45% to 26%) for "which ONE movie would you choose for the Academy Award for Best Picture?"

Also, a national survey of 1,000 adults (conducted 2/18 through 2/19) found The Curious Case of Benjamin Button leading Slumdog Millionaire (33% to 21%) for which film "would you give the Oscar to for best picture" (SurveyUSA).