Pollster.com

January 25, 2009 - January 31, 2009

 

FL: 2010 Senate (DailyKos-1/26-28)


DailyKos.com (D) / Research 2000
1/26-28/09; 600 likely voters, 4% margin of error
400 likely Democratic/Republican primary voters, 5% margin of error
Mode: Live Telephone Interviews

Florida

Favorable / Unfavorable
Meek (D): 18 / 7
Boyd (D): 14 / 9
Gelber (D): 12 / 3
Crist (R): 65 / 23
McCollum (R): 42 / 23
Rubio (R): 11 / 17

'10 Senate Democratic Primary
Meek 17%, Boyd 8%, Gelber 3%

'10 Senate Republican Primary
Crist 57%, McCollum 11%, Rubio 4%
McCollum 28%, Rubio 12%

'10 Senate General Election
Crist 49%, Meek 28%
Crist 52%, Boyd 26%
Crist 52%, Gelber 21%

McCollum 47%, Meek 31%
McCollum 48%, Boyd 28%
McCollum 50%, Gelber 22%

Meek 31%, Rubio 22%
Boyd 29%, Rubio 22%
Gelber 23%, Rubio 23%

(source)


Pre-Super Bowl "Outliers"

Topics: Outliers Feature

Gary Langer says the partisanship of House Republican's is consistent with the policy preferences of Republican voters.

Ruy Teixeira sees a big mandate for change.

Jon Cohen spots a big gap in the favorability ratings of the Democratic and Republican parties.

Tom Jensen chews over Gallup's 50-state party identification release.

Nate Silver ponders whether Roland Burris can win reelection.

Brendan Nyhan thinks Reagan had a bigger mandate than Obama.

David Winston says the stimulus plan is rooted in ideas from the 1930s.

Mark Mellman looks forward to 2010.

David Hill thinks Obama will become a force for limiting abortion.

Chris Bowers says public opinion will turn against Afghanistan escalation.

Greg Sargent shares polling conducted for the Democratic health reform group Health Care for America Now.

John Sides argues that partisans should release more data.

Survey Practice dives deep into "zero bank" bias (for those who really, really enjoy the intricacies of methodology).

New York Post has best. polling. headline. ever. (via Crowley).

Carl Bialik sees a rise in usage of "tipping point" ("Outliers" too! Thank you, Malcolm).

Jennier Agiesta reviews the political partisanship of Superbowl contender states Pennsylvania and Arizona.

Andrew Gelman says sports fans and non-fans share similar political ideologies.

Jay Cost breaks down the Steelers' stats.

And Eric Dienstfrey rounds up the Super Bowl poll questions.


Super Bowl Roundup


55% think the Steelers will win the Super Bowl, but only 44% want them to win. (Rasmussen)

Democrats are more likely to support the Steelers; Republicans are more likely to support the Cardinals. (FOX)

61% of adults plan on watching the Super Bowl. (Gallup)

Half-time performer Bruce Springsteen has a net favorable rating of +17%. (Rasmussen)

Americans with a post-graduate degree are "least likely to call football their favorite sport." (Harris)


US: National Survey (DemCorps-1/26-29)


Democracy Corps (D)
1/26-29/09; 1,000 likely voters, 3.1% margin of error
Mode: Live Telephone Interviews

National

State of the Country
27% Right Direction, 65% Wrong Track

Obama Job Approval
55% Approve, 20% Disapprove

Favorable / Unfavorable
Obama: 62 / 20
Democrats in Congress: 43 / 35
Republicans in Congress: 33 / 40

As you may have heard, President Obama says his first act will be to pass an economic recovery plan. From what you have heard about Obama's plan, do you favor or oppose his proposal?

    69% Support
    24% Oppose

If your Representative in Congress voted in favor of President Obama's economic recovery plan, would it make you more or less likely to support your Representative for reelection it two years, or does it make no difference?

    34% More likely
    20% Less likely
    43% No difference

(source)


The Limbaugh Brand

Topics: Barack Obama , Favorable Ratings , Rush Limbaugh

Do you think this week's back-and-forth between the Obama administration and Rush Limbaugh is an accident? Or did the Democrats welcome an opportunity to make the conservative talk radio host the symbol of opposition to Obama's stimulus proposal? A look at what little polling data exists on Limbaugh says that accident or not, Democratic strategists are probably happy to see Limbaugh's profile rise.

Let's go back to the beginning. On Friday, January 16, used his radio program to declare, "I hope Obama fails."

A week later, last Friday, in the midst of a private meeting, Obama urged Republican leaders on Capitol Hill to ignore Limbaugh saying, "you can't just listen to Rush Limbaugh and get things done."

Then this Monday morning, the topic came up in the of the daily White House press briefing. As Chris Cillizza reported,

White House press secretary Robert Gibbs offered another sly provocation of Limbaugh; asked to expand on Obama's comments about the talk show host, Gibbs demurred and then added: "Tell [Rush] I said hi."

This week, Limbaugh has happily engaged on the topic, both on the air and in a Wall Street Journal op-ed yesterday. Americans United For Change, a Labor backed group, is now running radio advertisements in Pennsylvania, Ohio and Nevada repeating Limbaugh's "I hope he fails," pointing out that "every Republican voted with Limbaugh" and asking whether "our Senator, ____, will side with Limbaugh too."

For much of the week, the Obama-Limbaugh fued has been a prime topic of conversation on the cable news networks, with much speculation that Obama erred in mentioning Limbaugh by name last Friday. Was it?

Oddly, very few polling organizations have asked questions about Limbaugh. I could find just one public pollster -- Democrat Stan Greenberg and his Democracy Corps project -- that tested Rush Limbaugh's favorable rating during 2008. The most recent was on a survey sponsored by the liberal group Campaign for America's Future that fielded immediately after the election (November 4-5).

20080130_Limbaugh.png

As the table above shows, Limbaugh's unfavorable rating among all voters (59%) is higher than any other Republican except George W. Bush, although Sarah Palin (47%) and New Gingrich (48%) earn negative marks that fall within the margin of sampling error of Limbaugh. Nonetheless, it should be clear that Limbaugh has a different image than "conservatives" (34% unfavorable), Fox News (35%) or John McCain (37%).

One reason is the way Republicans perceive Limbaugh. In An earlier Democracy Corps survey, fielded in mid-October, showed that nearly a third of Republicans (32%) give Limbaugh a unfavorable (cold) rating, while just 44% rate him favorably.

We may never know whether Obama intended to raise this ruckus when he dropped Limbaugh's name a week ago. However, Robert Gibbs and the others in the White House certainly knew the question was coming on Monday and their reaction is telling. After Chris Cillizza asked his Republican sources whether Limbaugh's higher profile would be good or bad for Republicans, Ben Smith made this very good point:

"[B]ipartisanship" is as much a brand as any conceivable Washington reality. These House Republicans, as is traditional when a caucus shrinks, are more conservative, and in safer seats, than their predecessors. The notion that they'd wind up anything other than extremely rare allies of the Democratic President was always unlikely. Obama doesn't need their votes. But his visible, cable-television-grabbing bipartisan gestures are aimed at cementing his hold on that brand, and ensuring when Republicans and Democrats go their separate ways, Republicans are seen as the partisan ones.

It's not a particularly novel tactic, but it places the House Republicans in an uncomfortable spot. As Chris Cillizza wrote in a very smart piece today, their party is in danger of being defined as pure, intransigent, Rush-Limbaugh-style opposition, and Obama's visit to the Hill may give their image a further shove down that road.

My guess is that some Democratic sponsored focus group conducted this week confirmed that voters in the middle perceive Rush Limbaugh as the antithesis to the Obama "bipartisanship brand," and as such, are not unhappy to see Limbaugh's profile in this rfight rise.

PS: The Democracy Corps "feel thermometer" favorable rating may seem usual, but it is essentially the same question and format used on in-person surveys by the American National Election Studies since the 1950s. The text of the Greenberg/Democracy Corps question is as follows:

Now, I'd like to rate your feelings toward some people and organizations, with one hundred meaning a VERY WARM, FAVORABLE feeling; zero meaning a VERY COLD, UNFAVORABLE feeling; and fifty meaning not particularly warm or cold. You can use any number from zero to one hundred, the higher the number the more favorable your feelings are toward that person or organization. If you have no opinion or never heard of that person or organization, please say so.

They report numbers between 51 and 100 as favorable ("warm") and numbers between 0 and 50 as unfavorable ("cold").


IL: 2010 Senate (DailyKos-1/26-28)


DailyKos.com (D) / Research 2000
1/26-28/09; 600 likely voters, 4% margin of error
400 likely Republican/Democratic primary voters each, 5% margin of error each
Mode: Live Telephone Interviews

Illinois

Favorable / Unfavorable
Sen. Burris (D): 35 / 35
Schakowsky (D): 33 / 10
Giannoulias (D): 36 / 15
Kirk (R): 37 / 41
Roskam (R): 19 / 23

'10 Senate Democratic Primary
26% Burris, 12% Schakowsky, 11% Giannoulias

'10 Senate Republican Primary
27% Kirk, 17% Roskam

'10 Senate General Election
37% Burris, 30% Kirk
38% Burris, 25% Roskam

36% Schakowsky, 30% Kirk
37% Schakowsky, 25% Roskam

38% Giannoulias, 30% Kirk
38% Giannoulias, 25% Kirk

(source)


"Manipulating" Public Opinion


My colleague, Mark Blumenthal, has recently posted his reaction to an earlier post of mine, in which I suggested that most major media pollsters deliberately manipulate public opinion, in order to make it appear as though most of the public has an opinion on an issue. My examples included polls about the stimulus package, and how the public viewed the Democrats' control of the three branches of government.

 

In his critical remarks, Mark suggested that I was being a bit narrow in my view of public opinion and unfair in implying a nefarious motive on the part of the pollsters. In a later blog on the same issue, he noted that he had received reactions from two different pollsters, who did not want to reveal their names, one who works on campaigns and the other who works for the media. They provided somewhat different takes on his discussion of public opinion about the stimulus package, takes which I think tend to support my criticisms of media polls.

 

More about that in a moment. First, let me say that I appreciate Mark's well-considered criticisms, recognizing that they probably also reflect the views of many other practicing media pollsters (though let's hope not all). And I appreciate the opportunity that Mark offers for me to blog on this site about these issues, because I think such conversations are at the heart of the scientific enterprise - as does he. For that I am grateful.

 

As to the flaws in my concept of public opinion, I think Mark may misunderstand my recent focus on the lack of "no opinion" measures. Like Mark, I think that is just one part of measuring public opinion, but still a crucial one. Mark seems to agree, writing that "Yes, it is important to understand that many Americans lack a specific opinion on the 'economic stimulus' legislation per se, something stressed by too few pollsters. Still, that finding is just one part of 'public opinion' on this issue." (emphasis added)

 

I couldn't agree more. My view is that at on important policy matters, pollsters should measure at least three dimensions of public opinion: 1) direction of support (from support to opposition), including the magnitude; 2) intensity of views; and 3) the absence of a meaningful view on the matter, or non-opinion. In my recently book, The Opinion Makers, I elaborate more fully on this concept. (For the time being, I will ignore the oft-neglected measure of intensity.)

 

If measuring the direction of public opinion and measuring non-opinion are both important, why don't we find both measures in most polls? Look at the graph below - these are the poll results that Mark assembled from the various polls in his critique of my commentary. All of them measure direction of opinion, as we would expect, but only one attempts to measure non-opinion (NBC/WSJ). (Mark suggests that Rasmussen may have provided an explicit "no opinion" option, but Rasmussen's topline with the actual question shows it was a forced-choice format, with "no opinion" a volunteered option.)

 

(The graph below takes the difference between the percentage of people who support and the percentage who oppose the stimulus package, as described in the respective polls, which is then plotted as the "margin in favor" - since all polls showed more people in favor than opposed to the stimulus. The graph also shows the percentage of people without an opinion, as reported by each poll.)

 

0901_30 Reply to Mark B on Manipulating public opinion (Econ stim diff polls).png Mark acknowledges that too few pollsters stress the percentage of people who lack an opinion (see italicized part of his quotation above), and his (and my) concerns are amply illustrated in the graph. Only 1 percent have no opinion according to CNN, just 3 percent say ABC/WP, 4 percent says Ipsos, 7 percent say NBC/WSJ, and 11 percent to 12 percent say Gallup and Hotline. Those are hardly credible numbers, if we are referring to a specific stimulus package (rather than to the general idea of some kind of stimulus), as all of these results do.

 

Indeed, Mark says the consensus of the two pollsters who contacted him in the wake of his critique was that "both imply agreement on one thing: Most Americans know little about the 'economic stimulus plan,' except that the President and the Congress are talking about it." Mark also adds at the end of his commentary that "'tepid' support is about the right phrase to use" to describe public opinion on the issue.

 

If that is the case (and I tend to agree with it), how did these pollsters arrive at that conclusion? Certainly not by looking at Gallup, Hotline, Ipsos, NBC/WSJ, CNN or ABC/WP. All those pollsters suggest very few people unsure about the specific stimulus plan being considered by Congress, and very large majorities in favor. None of these pollsters suggested "tepid" support and widespread ignorance.

 

Mark also writes that besides measuring non-opinion, "reactions to new information are also important, as are the underlying values driving responses to all of the questions reproduced above." Again, I don't disagree, though he implies that I do. It is not mutually exclusive to measure non-opinion (which most pollsters fail to do) and also to measure reactions to new information or underlying values.

 

The question is why do most pollsters fail to measure non-opinion?

 

(Separately, why do most pollsters fail to measure intensity? That's also an important dimension, but I'll talk about the failure to measure intensity at a later time.)

 

My response is that most pollsters don't measure non-opinion in general because they don't want to reveal that a sizeable number of Americans don't have an opinion on important policy matters. Mark thinks I'm being unfair, and that I'm attributing nefarious motives to such pollsters.

 

Here's the dilemma: Mark and I both agree that non-opinion is a crucial part of measuring the public's position on policy matters. We also know that all the major media pollsters do measure non-opinion from time to time. So, why don't they measure non-opinion on such major issues as the stimulus? What criteria do they use to determine when they will, and when they won't, ask questions with an explicit "no opinion" option?

 

The simple answer is that the news media wouldn't find it interesting to constantly report that large segments of the population don't have a meaningful opinion about the major policy issues facing the country. That's why pollsters feed respondents information and asked forced-choice questions, all in an effort to reinforce the myth of a highly informed, rational, and engaged public. It's a far more newsworthy myth than the reality of a public with many people uninformed and unengaged on issues, and thus lacking meaningful opinions.

 

I'm not arguing that all people fit that category. I'm only arguing that pollsters and the media should be willing to admit the existence, and measure the size, of such large segments of the public, instead of manipulating respondents to come up with answers so that it will appear as though virtually all Americans have a meaningful opinion.

 

In 1942, Elmo Roper wrote in an essay for Fortune magazine, titled "So the Blind Shall Not Lead," that even then, less than a decade since the advent of modern polling, "the emphasis in public opinion research has been largely misplaced. I believe its first duty is to explore the areas of public ignorance."[1]


Exploring areas of public ignorance may not necessarily be the pollsters' first duty, but it is certainly an important duty they usually fail to perform.



[1] Elmo Roper, "So the Blind Shall Not Lead," Fortune, 25, No. 2, p. 102, cited in George Bishop, The Illusion of Public Opinion: Fact and Artifact in American Public Opinion Polls (Lanham, Maryland: Rowman & Littlefield Publishers, Inc., 2005), p. 6.

 


Fun with Order Effects

Topics: Fox News , Opinion Dynamics , Order effects , Timothy Geithner

A quick thought about the approval question asked about the confirmation of new Treasury Secretary Timothy Geithner on the new Fox/Opinion Dynamics poll (that we blogged earlier this afternoon). Take a look at the question asked just before it:

8. Do you think failing to pay part of your taxes for a few years is enough to disqualify someone from serving as Treasury Secretary and heading the Internal Revenue Service?

58% Yes
33% No
9% Don't know

9. Do you approve or disapprove of the Senate confirming Timothy Geithner as Secretary of the Treasury?

35% Approve**
44% Disapprove
21% Don't know

Gee, do you think that results for question #9 might have been different had it come before question #8?

Here's a friendly request to our readers that regularly conduct national omnibus surveys (and you know who you are): Could you try a split form experiment where half the respondents hear question #8 first and the other half hear question #9 first?   

The original version of this post mistakenly used the wrong answer categories for question #9


About those Dubious Polling Awards

Topics: David Moore , Dubious Polls Award , George Bishop

Let me start with the "my bad" portion of this entry: Three weeks ago, our colleague David Moore sent me an early draft of the "Dubious Polls Awards" commentary he co-authored with George Bishop. Moore asked for my comments, but in an oversight that speaks to my own poor management of an overflowing email inbox, I set the message aside without reading the attachment and soon forgot about it. He ultimately posted a summary here earlier today, with the more detailed version posted on stinkyjournalism.org. Had I read the draft, I would have given David feedback consistent with what follows. I apologize to David and our readers for that oversight, but I want to take this opportunity to air the issue publicly and allow readers to react and comment.   

I make no apologies, on the other hand, for giving David Moore (and by extension, George Bishop), the opportunity to blog here at Pollster. As I noted earlier this week, David brings to this endeavor a long career in the field of survey research, as an author, an academic and a former managing editor of the Gallup Poll. George Bishop is one of the most respected academic survey researchers, and though his perspective is sometimes at odds with others in the field, his work is something any serious pollster should know (particularly his book, The Illusion of Public Opinion ). If Moore and Bishop are willing to to act as provocateurs and criticize the most respected voices in the field, fine. They have the expertise to do so with authority, and constructive criticism has always been part of our mission.

I also believe that blogging works best when edited least. Holding back posts for review and revision kills the spontaneity and give-and-take that make blogging work. As Andrew Sullivan has written, readers are his best editors. "E-mail seemed to unleash their inner beast. They were more brutal than any editor, more persnickety than any copy editor."

The only rule I have tried to set for contributors here at Pollster is to follow my tone: Avoid name calling and gratuitous snark and, above all, be fair.

My problem with the Dubious Polls summary -- and the feedback I should have given Moore and Bishop -- is that it offers far too much snark and name calling with just a smattering of the smart context that they are well equipped to provide (and do a better job providing in the longer version on stinkyjournalism.org). It is also, in places, less than fair.

Consider, for example, their "top award, earning five crossed fingers:"

[It goes to] all the major media polls for their prediction of Giuliani as the early Republican frontrunner. Collectively this group, beginning more than one year prior to the first statewide electoral contest in Iowa, relentlessly, and without regard for any semblance of political reality, portrayed Rudy Giuliani as the dominant Republican candidate in a fictitious national primary.

It is certainly true that most public pollsters reported results showing Giuliani leading, consistently, throughout most of 2007, on questions that asked Republican identifiers nationally to state their preference for the Republican nomination. And it is also true that far too many journalists and pundits (and some pollsters) looked at these early results, showing Giuliani with the support of just 30% of Republicans nationally, and wrongly assumed or predicted that the former New York mayor had some sort of lock on the Republican nomination.

If Moore and Bishop had argued in their summary that we should have paid more attention to polls in New Hampshire and Iowa than nationally,or that pollsters should have done more to caution poll consumers against reading too much into those early Giuliani leads, I would agree (and did, here and here, back in August 2007). I also agree that any "predictions" of a Giuliani triumph based on those 2007 horse race polls alone ignored many political realities, including the fact that we do not hold a single-day, national presidential primary.

But is it fair to characterize as a "prediction" every horse race result released by the ten organizations Moore and Bishop list? Is it fair to use the phrase "crossed fingers" -- words that imply deliberate dishonesty -- to depict the release of those results? It feels unfair to me.

Obviously, this site is not my exclusive domain. Our goal for Pollster.com is to present a wide variety of poll and survey related content from many different authors, and we do not expect every front page contributor to agree or speak with one voice. On the question of tone, however, I want to hear from you. Please read over the Dubious Polls piece here, and the longer version on stinkyjournalism.org. What advice would you offer -- to David Moore or to me -- for future contributions?

Please feel free to comment below or email me directly. I will try to post excerpts from email in a future post (please stipulate if you prefer that your comments to remain totally off the record).


US: National Survey (FOX-1/27-28)


FOX News / Opinion Dynamics
1/27-28/09; 900 registered voters, 3% margin of error
Mode: Live Telephone Interviews

National

Obama Job Approval
65% Approve, 16% Disapprove

Congressional Job Approval
40% Approve, 46% Disapprove

How many campaign promises do you think Barack Obama will accomplish during
his presidency -- almost all of them, most of them, only some of them or hardly
any of them?

    13% Almost all
    35% Most
    39% Some
    10% Hardly any

Do you approve or disapprove of the Senate confirming Timothy Geithner as
Secretary of the Treasury?

    35% Approve
    44% Disapprove

Do you think the military prison at Guantanamo Bay should be closed, or
not?

    47% Yes, should be closed
    45% No, should not be closed

(story, results)


NY: Paterson, Gillibrand (Siena-1/25-27)


Siena Research Institute
1/25-27/09; 622 registered voters, 3.9% margin of error
Mode: Live Telephone Interviews

New York State

Favorable / Unfavorable
Gov. Paterson (D): 54 / 30
Sen. Schumer (D): 66 / 23
Sen. Gillibrand (D): 30 / 13
Giuliani (R): 60 / 35
Clinton (D): 67 / 28

'10 Senate Republican Primary
Giuliani 69, King (R) 16

'10 Senate General Election:
Gillibrand 44, Giuliani 42
Gillibrand 46, King 23

(source)


George Bishop's and David Moore's 2009 Top Ten "Dubious Polling" Awards


My colleague at the University of Cincinnati, George Bishop, and I have launched what we expect to be an annual listing of the Top Ten "Dubious Polling" reports for the previous year. Posted on Stinky Journalism.Org, we intend this as a satirical look at some of the practices of the major media pollsters.

 

As the opening paragraph notes: "Every year, poll watchers are confronted with poll results and commentary that defy either logic or science, often raising question about the very utility of polls. Typically, the problems are not with the method of conducting polls, but with the pollsters themselves - as they focus on what they believe is entertaining and appealing to the audience rather than an accurate reflection of public opinion. In the process, pollsters manipulate public opinion or write commentary that makes a mockery of what the public is really thinking."

 

Each award is ranked, from a low of one set of crossed fingers to a high of five sets. Pollsters generally know in their hearts when all is not right with their polls, but they (figuratively) cross their fingers and hope that no one notices anything amiss. The five crossed-fingers icon is the ultimate in wishful thinking, perhaps the equivalent of football's "Hail Mary pass" for the truly untrustworthy poll.

 

Our top award - earning the five crossed fingers - goes to all the major media polls[1] for their prediction of Giuliani as the early Republican frontrunner. Collectively this group, beginning more than one year prior to the first statewide electoral contest in Iowa, relentlessly, and without regard for any semblance of political reality, portrayed Rudy Giuliani as the dominant Republican candidate in a fictitious national primary.

 

Other "Dubious Polling" awards are:

·        Loopiest Poll Award: Pew Research Poll, for weekly pre-election polls in October that showed wild swings in Obama's lead.

·        Shooting Yourself in the Foot Award: Gallup Poll, for publishing two polls on Feb. 25, 2008, that contradicted each other.

·        Over-the-Top Gloating Award: Gary Langer, polling director of ABC, for writing that "What I liked best about the final New Hampshire pre-election polls [which erroneously predicted Obama to win] is that I didn't do any of them" - cleverly completing his polling in the Granite State far enough away from the election to avoid having his results compared with the election outcome.

·        180 Degree Award: CBS News/New York Times and USA Today/Gallup polls, for coming to opposite conclusions about the controversy over Rev. Jeremiah Wright.

·        Waiting for Godot Award: The American Association for Public Opinion Research Committee that still has not issued a report on the erroneous predictions in the N.H. Democratic Primary.

·        Who Knows? Award: Pew Research, ABC News/Washington Post, and Los Angeles Times/Bloomberg polls for contradictory conclusions about public support for Wall Street bailout.

·        Wake-Me-Up-When-It's-Over Award: NPR, Kaiser Family Foundation, and the Harvard School of Public Health Survey for a vague 131-word question.

·        Flip-Flop Award: CNN for two December polls that showed opposite results of the public's support for the auto bailout.

·        For Sale! Award: Peter D. Hart Research Associates, for their General Motors-sponsored poll that found (surprise! surprise!) overwhelming public support of auto industry bailout.

 

For a full description and rationale for the awards, go to Stinky Journalism.Org.



[1] These include polls by the Associated Press, ABC News/Washington Post, CBS News/New York Times, CNN, FOX, NBC News/Wall Street Journal, USA Today/Gallup, Newsweek, the Los Angeles Times/Bloomberg, and Pew Research.

 


Taylor: Do Pollsters Need a Code of Ethics to Prevent "Hired Guns Polls" Designed to Get the Answers the Clients Want?


Today's guest pollster contribution comes from Humphrey Taylor, who has served as chairman of The Harris Poll, a service of Harris Interactive, since 1994.

In the late 1970s, Louis Harris & Associates (one of the two firms that merged to form Harris Interactive in 1996) was commissioned to conduct a survey of American attitudes to allowing oil companies to drill for oil in wilderness areas that were off-limits. Unfortunately neither Lou Harris nor any of his senior colleagues knew anything about this survey until we read the results in the media -which showed strong public support for opening up wilderness areas for drilling, quite contrary to the findings of other polls by Harris and other firms.

When we looked at the questionnaire it was immediately obvious that this was a particularly egregious example of a "hired gun poll" designed to get the answers the client wanted, a survey designed to mislead rather than to inform policy makers about public opinion.

Lou Harris, to his great credit, publicly disowned the survey and said that the findings did not reflect public opinion. Soon afterwards, he received a phone call from an irate Senator Stevens of Alaska, who was apparently close to our clients, trying unsuccessfully to persuade Lou to back down. And the clients never paid the bill.

Were I a lobbyist with a strong point of view, I (or any competent pollster) could without much difficulty write a series of questions which might be technically acceptable, but which - taken together or separately - would mislead the media and many legislators into believing that public support for my client's position was much stronger than it really is. And here I'm not talking about bad samples or manipulating data to get the answers I want, only about the design of the questionnaire itself.

Over the years I have read many published surveys that fitted the description "hired gun polls." They are often easy for pollsters to spot because the results are surprising and are strikingly different from the results from other polls. Furthermore, they provide powerful arguments that can be used by those who paid for the survey to lobby government, to influence elected officials and to generate favorable publicity. However, it is much harder for most people, including policy makers and influentials, who are not themselves pollsters to recognize, and discount, those "hired gun polls." And this can be made more difficult when the funding source - which may be a company, a trade association, a public relations firm, an NGO or an advocacy group - is hidden behind a supposedly independent third party.

This would not matter if polls had no influence, but I strongly believe that polls sometimes influence the political agenda, policy makers, regulators and the way the media cover issues. At the risk of sounding pompous, I believe all pollsters whose polls are released to the media and the public have a moral obligation that they inform and do not mislead. Unfortunately, hired gun polls are designed to mislead.

In the polling community, some polling firms are seen as particularly bad practitioners of hired gun polling. Some will defend their right to ask whatever questions they and their clients want. Most major polling firms probably try to avoid hired gun work, but in an imperfect world it is not always easy to spot them early enough to prevent them.

Those who are not pollsters may be puzzled by this whole issue. They may believe that the public opinion is what it is and that as long as a representative sample is surveyed the replies will reflect public opinion. Pollsters know better. Different questions on the same issue can produce very different and apparently contradictory results, and questions asked earlier in a survey can have a big impact on questions that are asked later. It is not difficult to write questionnaires that greatly increase the number of people who give a particular response. Fortunately the reputation of our organizations can be hurt if we are seen as to be doing this, and our business may suffer. But, sadly, some firms that have a track record of doing hired gun polls are still in business.

Some pollsters' clients may be puzzled. Once in Latin America, I asked a presidential candidate why he had published polls showing him in the lead, when no other polls did so (he was soundly defeated). His immediate and refreshingly honest reply, "If I pay for a poll, I should be able to get the answers I want and am paying for." He is not unique.

To address this problem Harris Interactive has a set of guidelines and procedures that I summarize below. I would be delighted to hear that other survey firms have similar rules. And I would be thrilled if the leading polling firms could agree on a code of ethics based on similar principles.

We pollsters need to put our house in order so that we do not have to defend polling that is morally and ethically indefensible.

Hired gun polls damage the credibility and reputation of polling and pollsters and, I am glad to say, can - when spotted and criticized - be damaging to the firms that do them and to their clients. A much more serious effect is that they can mislead and misinform policy makers, opinion leaders and the media.

The leading polling organizations have a code but it is a code of disclosure, not a code of ethics. It is necessary but not sufficient. Many years ago they came together to found the National Council on Public Polls (NCPP) and agreed to a code of disclosure that describes what they must publish whenever they release a new poll (they must describe the universe, the methodology, the fieldwork dates and the relevant questions, and who commissioned the research). But complying with the NCPP code of disclosure does not do much to help the readers to recognize a hired gun poll, that is intended to mislead rather than to inform.

Maybe the time has come for pollsters to agree on a code of ethics that would inhibit them from conducting surveys that they know are designed to get the answers the clients want. This is not an argument against conducting and publishing any polls for interest groups and advocacy organizations. They fund many valuable and useful surveys. Rather it is a plea that pollsters do everything they can to ensure that such polls are sufficiently comprehensive and are fair and balanced (although those words seem to have taken on a new meaning) that they do not mislead.

Note: This document provides a summary if the ground rules that bind us and our clients who commission surveys for public release.


More on the Stimulus

Topics: Economic stimulus , Measurement , Stimulus

I received two different reactions from two pollsters who shall remain nameless this morning to yesterday's post on public reaction to the "economic stimulus program" being debated in Congress. Their comments provide two somewhat different takes on how to think about public opinion and the stimulus plan.

The first comes from a campaign pollster:

Any idea how voters feel about 200 million to resod the Mall or money for “urban stabilization” or tax refunds for individuals who don’t pay/owe taxes? Just a rhetorical way to point out that the macro-programs almost always generate more supportive poll numbers than polls about particular parts of them.

The second from a media pollster:

My overall take is support for the plan is tepid, but people are willing to give the President the benefit of the doubt. It’s also clear that the spending side of the equation is big enough to be costing support for the plan. Obama is a very gifted politician and playing his hand very well.

Both imply agreement on one thing: Most Americans know little about the "economic stimulus plan," except that the President and the Congress are talking about it.

Beyond that their comments suggest two important dynamics that will shape where opinions on the "economic stimulus plan" will end up several months down the road: First, how will the news media describe and cover the plan? Will it focus on individual line items (like re-sodding the Mall) or on the larger "macro" goals for the country? Second, what role will leadership (that is, trust in either Obama or the Republicans in Congress and the persuasiveness of the arguments they make) play in shaping opinions?

We do have a sense of how Americans react to some specific items that may be included in the plan (see the analysis from both the (Washington Post and ABC News and Q30 on the NBC/Wall Street Journal poll) and about their continuing fears of excessive government spending (see Q17 and Q27 on the NBC/WSJ poll). Of course, we also know a lot about perceptions of Obama and the Congress. Which of these, if any, will be most important in shaping the sure-to-evolve opinions on the stimulus plan? Your guess is probably as good as mine.

So, going back to the results in yesterday's post, we know something about where public opinion on "the economic stimulus" is now, and "tepid support" is about the right phrase to use. Knowing where public opinion may be in a month or in six months, however, is very hard to predict.


US: 50 State Party ID (Gallup-2008)


In the first of a four-part series titled "State of the States," Gallup released party affiliation for all fifty states based on Gallup Poll Daily tracking data collected throughout 2008. Note that Gallup added partisan-leaning independents to the percentage that identify with either the Republican or Democratic parties in order to make the statewide data more comparable.

All told, 29 states and the District of Columbia had Democratic party affiliation advantages of 10 points or greater last year. This includes all of the states in the Northeast, and all but Indiana in the Great Lakes region. There are even several Southern states in this grouping, including Arkansas, North Carolina, and Kentucky.

An additional six states had Democratic advantages ranging between 5 and 9 points.

In contrast, only five states had solid or leaning Republican orientations in 2008, with Utah, Wyoming, Idaho, and Alaska in the former group, and Nebraska in the latter.

The most balanced political states in 2008 were Texas (+2 Democratic), South Dakota (+1), Mississippi (+1), North Dakota (+1), South Carolina (even), Arizona (even), Alabama (+1 Republican), and Kansas (+2 Republican).

The full analysis can be found here.


Economic Stimulus and the Many Faces of "Public Opinion"

Topics: Divergent Polls , Economic stimulus , Measurement , Professor M

Last week our colleague David Moore highlighted some contradictory results on the issue of the proposed economic stimulus program to illustrate, as he put it, "how easy it is for pollsters to manipulate public opinion into something different from what it really is." He pointed out that a question asked by Gallup produced just 11% with no opinion, while a similar question posed by the NBC/Wall Street Journal poll produced 30% in that category when it prompted respondents to say when they "do not have an opinion either way."

David is nothing if not provocative, and he comes to his opinions from a long career in the field of survey research. For those reasons we are glad he has become a regular contributor. I also agree with him that pollsters too rarely report probes of overall awareness of specific proposals, or conversely, the reluctance of respondents to offer an opinion when asked questions about complex public policy issues. More often than not, issue questions on media polls measure the way the public reacts to the information presented rather than a pre-existing opinion on the proposal being discussed. That habit, as Moore argues, tends to exaggerate our sense of the public's engagement in matters of public policy. But I part company with David when he implies a nefarious intent, arguing that pollsters "are generally not interested in realistic measures of public opinion," and choose instead to deliberately "create" or "manipulate" public opinion.

By way of explanation, consider all of the questions asked over the last few weeks that attempt to gauge support for the proposed economic stimulus legislation. I have ranked the table below on the percentage expressing support (all are from January, follow the links for details on specific dates, sample sizes and reported margins of error).

NBC/Wall Street Journal: Do you think that the recently proposed economic stimulus legislation is a good idea or a bad idea? If you do not have an opinion either way, please just say so. 43% good idea 27% bad idea 30% no opinion/ unsure
Rasmussen Reports (automated): Do you favor or oppose the economic recovery package proposed by Barack Obama? 45% favor 34% oppose 21% unsure
Gallup: Do you favor or oppose Congress passing a new $775-billion dollar economic stimulus program as soon as possible after Barack Obama takes office? 53% favor 36% oppose 11% unsure
Hotline/Diageo (asked of half sample): In general, do you support or oppose Congress and the President passing an $825 billion economic stimulus plan to jumpstart the economy, even if it means increasing the federal deficit in order to do so? 54% favor 34% oppose 12% unsure
IPSOS/McClatchy: With the current stimulus package being considered, the total cost of the economic stimulus could total one trillion dollars. Is spending this amount of money definitely necessary, probably necessary, probably not necessary or definitely not necessary? 55% necessary 41% necessary 4% unsure
NBC/Wall Street Journal (follow-up to NBC question above): When it comes to the economic stimulus plan proposed by the Obama administration, which of these two statements comes closer to your point of view? Statement A: The economic stimulus plan is a good idea because it will help make the recession shorter, get people back to work, and provide money for transportation, education, and Medicaid programs. Statement B: The economic stimulus plan is a bad idea because it will do little to shorten the recession, the jobs are temporary, and it will significantly increase the deficit. 57% good idea 36% bad idea 7% unsure
CNN/ORC: Would you favor or oppose a proposal to attempt to stimulate the economy by increasing federal government spending on construction projects and economic assistance to some Americans by about eight hundred billion dollars? 58% favor 40% oppose 1% unsure
IPSOS/McClatchy : Do you think that an economic stimulus package is necessary to improve the current state of the economy, or not? 62% necessary 32% not 6% unsure
Hotline/Diageo (asked of half sample): As proposed, the $825 billion economic stimulus plan would include $550 billion in new spending for alternative energy technology, roads and bridges, state governments and local school districts, and increasing benefits for the unemployed. The remaining part of the stimulus package contains $275 billion in tax cuts and credits for individuals and for business to help generate more jobs. In general, do you support or oppose Congress and the President passing this $825 billion economic stimulus plan to jumpstart the economy, even if it means increasing the federal deficit in order to do so? 67% favor 27% oppose 6% unsure
ABC News/Washington Post - Would you support or oppose new federal spending of about 800 billion dollars on tax cuts, construction projects, energy, education, and health care to try to stimulate the economy? 70% support 27% oppose 3% unsure
CNN/ORC (follow-up to CNN question above): And if that economic stimulus plan also included tax cuts for individuals and businesses, would you favor or oppose that proposal? 71% favor 28% oppose 1% unsure

When you compare the results this way, a few obvious patterns emerge:

1) When asked whether they favor or oppose an "economic stimulus" plan that would cost $800 billion or so (give or take a hundred million), Americans generally express support in the mid-50-percent range.

2) When pollsters also provide an explicit "do not have an opinion" option, as NBC/Wall Street Journal and (presumably) Rasmussen do, support falls to the mid-40 percent range and (on the NBC/WSJ poll at least) opposition also falls proportionately.

3) When the questions provide more information on how the $800 billion (or so) will be spent, usually specifying a combination of tax cuts and transportation, education and energy projects," support grows to mid-60 low-70 percent range.

4) Only one question -- again from NBC/Wall Street Journal -- poses explicit arguments for and against the proposal, and it produces a slightly higher level of support (57%) than the first category of questions that mention only the overall price tag and omits a specific prompt for "no opinion."

5) Every question shows net support for the proposal.

David Moore argues, in effect, that only the second category of question provides a "realistic" measure of public opinion. There we disagree. Yes, it is important to understand that many Americans lack a specific opinion the "economic stimulus" legislation per se, something stressed by too few pollsters. Still, that finding is just one part of "public opinion" on this issue. Reactions to new information are also important, as are the underlying values driving responses to all of the questions reproduced above.

Think about it this way. Suppose you had the chance (and the time and sheer stamina) to sit down and ask a random sample of Americans, one at a time, to share their opinions about what the government in Washington should be doing to "stimulate" the economy. A substantial number, though a minority, would have strong awareness of the details of the plans being debated and would offer highly informed, strongly held opinions. A smaller number, I'm guessing, would offer little more than a blank stare, having no idea what plans are being debated and would express no real opinion about what the government should do even when provided some vague details on the options being considered.

The vast majority, however, would fall somewhere in between. The would offer some opinion about what the government should do, even if they know little about the "economic stimulus" proposal now being debated in Washington. We know (from questions on the recent ABC/Washington Post or the Pew Research Center surveys) that at least three out of four Americans want their government to do something to strengthen the economy. Some are ready to trust (or distrust) whatever the new Obama administration is proposing, on the basis of their trust (or distrust) of our new president alone. Others will have general attitudes about government spending, taxes or specific priorities they want addressed or about other players in the debate.

And the important point is that all of this defines "public opinion" on the issue of economic stimulus. Poll analysts and journalists get in trouble when we try to boil all of public opinion on any legislative proposal down to a a single question, when we try to pick the most "correct" question when their wording differs (see the comments from my old friend Professor M here and here) or when we forget that survey questions sometimes tap both pre-existing opinions and reactions to new information.

PS: To be clear, the polls linked to above asked many questions on the economy and economic stimulus, not just the one or two from each I included above. The Polling Report also has a helpful compilation.


US: Obama, Stimulus (Hotline-1/21-24)


Diageo / Hotline / Financial Dynamics
1/21-24/09; 800 registered voters, 3.5% margin of error
Mode: Live Telephone Interviews

National

Favorable / Unfavorable
Obama: 76 / 15

Obama Job Approval
63% Approve, 9% Disapprove

Direction of U.S.
31% Right Direction, 52% Wrong Track

In general, do you support or oppose Congress and the President passing an $825 billion economic
stimulus plan to jumpstart the economy, even if it means increasing the federal deficit in order to do
so? And is that strongly or somewhat?
(n=401)

    54% Support
    34% Oppose

As proposed, the $825 billion economic stimulus plan would include $550 billion in new spending
for...alternative energy technology, roads and bridges, state governments and local school districts,
and increasing benefits for the unemployed. The remaining part of the stimulus package contains
$275 billion in tax cuts and credits for individuals and for business to help generate more jobs.

In general, do you support or oppose Congress and the President passing this $825 billion economic
stimulus plan to jumpstart the economy, even if it means increasing the federal deficit in order to do
so? And is that strongly or somewhat?
(n=399)

    66% Support
    27% Oppose

(release, data; Hotline (subscription) part 1, part 2; Hotline On Call part 1, part 2, part 3, video)


NY: 2010 Sen, Gov (Marist-1/26)


Marist College
1/26/09; 611 registered voters, 4% margin of error
Mode: Live Telephone Interviews

New York State

Favorable / Unfavorable
Sen. Kristen Gillibrand (D): 41 / 11

Gov. David Paterson (D) Job Approval
46% Excellent/Good, 48% Fair/Poor

Pres. Barack Obama (D) Job Approval
60% Excellent/Good, 22% Fair/Poor

'10 Senate General Election
Gillibrand 49%, Peter King (R) 24%
Gillibrand 44%, George Pataki (R) 42%

'10 Governor General Election
Michael Bloomberg (i) 47%, Paterson 41%
Rudy Giuliani (R) 47%, Paterson 46%

Compared to your views, do you think Kirsten Gillibrand is too liberal, too conservative, or about right?

    10% Too Liberal
    10% Too Conservative
    42% About Right

Kirsten Gillibrand has an "A" rating from the National Rifle Association. Does this make you more likely to vote for her for U.S. Senate in 2010, less likely, or does it not make a difference to your vote?

    18% More Likely
    25% Less Likely
    54% No Difference

(source)


NYC: 2009 Mayor (NY1-1/20-25)


NY1 / Baruch College Survey Research
1/20-25/09; 535 registered voters
Mode: Live Telephone Interviews

New York City

'09 Mayor Democratic Primary
Anthony Weiner (D) 31%, William Thompson (D) 22%, Tony Avella (D) 4%

'09 Mayor General Election
Mayor Michael Bloomberg (i) 45%, Thompson 32%
Bloomberg 43%, Weiner 36%

(source)


CO: 2010 Senate (PPP-1/23-25)


Public Policy Polling (D)
1/23-25/09; 959 registered voters, 3.16% margin of error
Mode: IVR

Colorado

Favorable / Unfavorable
Michael Bennet (D): 33 / 21
Scott McInnis (R): 32 / 32
John Suthers (R): 36 / 24
Bill Owens (R): 49 / 41
Tom Tancredo (R): 40 / 44

2010 Senate
Sen. Bennet 43%, McInnis 37%
Bennet 40%, Suthers 34%
Owens 44%, Bennet 41%
Bennet 48%, Tancredo 39%

(source)


NYC: 2009 Mayor (Quinnipiac-1/20-25)


Quinnipiac University
1/20-25/09; 1,216 registered voters, 2.8% margin of error
796 registered democrats, 3.5% margin of error
Mode: Live Telephone Interviews

New York City

'09 Democratic Primary for Mayor
Anthony Weiner 40%, William Thompson 23%

'09 General Election for Mayor
Mayor Bloomberg 50%, Weiner 35%
Bloomberg 50%, Thompson 34%

Favorable / Unfavorable
Bloomberg: 66 / 28
Weiner: 34 / 10
Thompson: 25 / 69

Bloomberg Job Approval
69% Approve, 25% Disapprove

(source)


Twittering "Outliers"


Ordinarily, I try to save my "outliers" entry -- with links to various polling or pollster related articles that I don't otherwise blog about in detail -- until week's end. However, a handful of interesting items appeared over the weekend, and I lost much of this day dealing with auto repair and a trip to DC's always tedious DMV auto inspection facility. So without further ado, some Monday afternoon "outliers."

  • Politico's Carrie Budoff Brown reports that both sides are "wielding poll-tested words" in the debate over the economic stimulus bill (see my take on a similar story earlier this month).
  • RCP's Jay Cost returned to blogging after a two-month absence with links to a four-part analysis of the 2008 election (one, two, three, four) that I had overlooked. Welcome back, Jay.
  • Lee Sigelman, a political scientist and Monkey Cage blogger, posts an intriguing question: Did the failure of political science research on race "to predict that voters in Iowa etc. would support a black man with the middle name of Hussein for president, especially over a Clinton and/or a war hero" mean that "political scientists screwed up again in our ability to understand and analyze contemporary politics?"
  • Andrew Gelman, after reading Nate Silver's take on the urban-ness of Obama's victory, reposts some graphs and analysis of urban rural voting patterns in 2008. His take: The Democratic gains in urban counties were not unique to 2008 and "the large-county/small-county differential in Obama's gains was particularly strong in the south and did not occur at all in the northeast."
  • Pollster Fred Yang, partner with Garin Hart Yang Research Group (a part of Hart Research Associates), has been identified by AP as the mysterious "Advisor B," tape recorded by a federal wiretap discussing with Governor Rod Blagojevich what to do with Barack Obama's Senate seat. Separately, the Washington Post reports that Geoff Garin, president of the firm, says Yang "is cooperating with the U.S. attorney's office, and I have no reason whatsoever to believe that he is a target of the investigation in any way."

Fred has been a friend for many years, even though we were often competitors. In a profession (political consulting) with more than its share of slippery, cut-throat characters, he has always been one of the most honest, collegial and well-respected. I would be stunned if he turns out to be more in this case than the victim of an out-of-control client.

Finally, for those who "tweet," I have made updates to Twitter more than an every-once-in-a-while habit (what is Twitter, you ask? click here). Under the name "MysteryPollster," I post links to my blog entries here and the occasional musing about things off-topic to Pollster.com. I would be honored to have you as a "follower."


NY: 2010 Gov (Siena-1/20-23)


Siena Research
1/20-23/09; 627 registered voters, 3.9% margin of error
Mode: Live Telephone Interviews

New York State

Favorable / Unfavorable
Paterson: 60 / 23
Obama: 81/ 10

Paterson Job Approval
51% Excellent / Good, 45% Fair / Poor

'10 Gov Dem Primary: Paterson (D-i) 35%, Cuomo (D) 33%
'10 Gov General Election: Paterson 44%, Giuliani (R) 42%
'10 Gov General Election: Cuomo 48%, Giuliani 39%

(source)


NY: Paterson, Gillibrand (Quinnipiac-1/23-25)


Quinnipiac University
1/23-25/09; 1,047 registered voters, 3% margin of error
Mode: Live Telephone Interviews

New York State

Do you approve or disapprove of the way Governor Paterson handled the process of selecting a new United States senator to fill Hillary Clinton's senate seat?

    44% Approve
    42% Disapprove

Do you approve or disapprove of Governor Paterson's selection of Kirsten Gillibrand to fill Hillary Clinton's senate seat?

    46% Approve
    30% Disapprove

Who do you blame for the controversy surrounding Caroline Kennedy's failed senate bid - Governor Paterson and his aides or Caroline Kennedy and her aides?

    15% Paterson and aides
    49% Kennedy and aides

(source)


 

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR