September 16, 2008
More on Message Testing and "Push Polls"
I had been meaning to write about two new reports of unusually ugly "message testing" polls that have popped up in recent days. As usual, journalists who should know better have reached for the "push poll" label, which is not quite right. These calls do not appear to fall into that category, though as in previous cases, the surveys are pretty ugly nonetheless.
Last week, Marc Ambinder reported on calls received in Ohio and Michigan from the Opinion Access Corporation that tested negative statements about the radical views of Obama's "spiritual advisor" and presumably slanted renderings of some of Obama's votes as State Senator. A DailyKos reader named RachelMo reported receiving the same call.
This week, Jonathan Cohn details of a survey call he received apparently aimed at Jewish voters that included a laundry list of incendiary statements involving Obama and Israel. Separately, Ben Smith reported on very similar sounding calls received by Jewish voters in New Jersey and Pennsylvania.
From what I can tell, all of these calls involved live interviewers and long interviews asking a variety of questions, including standard favorable ratings, issue questions and demographic items, as well as the battery that asked for reactions to the negative statements about Barack Obama. As such, they sound more like "message testing" -- albeit very ugy message testing -- than traditional "push polls."
Some background: A true "push poll" is not a poll at all, but usually a very brief call -- the modern version is typically recorded and automated -- that isn't intended to measure anything. Instead, the purpose is to communicate a (usually) scurrilous message to as many voters as possible. Real push polls are very short and aim to reach tens, sometimes hundreds of thousands or even millions of voters. These calls are dressed up to sound like a poll in order to (a) fool the recipients into listening and (b) add credibility to the ugly messages they contain.
Message testing is different, though those differences may seem semantic to the casual observer. Virtually all campaigns ask message testing questions on their benchmark surveys. Sometimes the messages tested are positive, sometimes negative. Some pollsters will repeat their vote preference question after testing messages, because they want to see whether their message will change opinion and, if so, with what voters. In that context, they are interested in how much they can "push" opinions, but as market research for paid ads, direct mail and the campaign messages.
Most of the time the messages tested -- positive and negative -- will tend to mirror the heated, one-sided rhetoric that we hear from candidates and their campaigns every day. Sometimes, however, the messages are extreme and offensive to those on the receiving end of the call, as was the case with the two latest poll stories.
The brain-dead way to approach these stories is to argue over whether the calls amount to a "push poll." As a campaign pollster, I helped design hundreds of surveys with similar tests of messages. So trust me when I say that all campaigns -- including the Obama campaign -- test positive and negative messages in their surveys. As I've written many times before, conducting a message testing poll does not absolve the pollster and the campaign from ethical obligations. The issue is not whether the pollster is trying to "push" the opinions, but whether they are telling the truth and treating their respondents with fairness and respect.
They way I wish reporters would approach these stories is to focus less on the "is-this-a-push-poll" angle and more on evaluating and debunking the charges they include.
See more of our past coverage of push polls and message testing, Stu Rothenberg's must read on the subject and the statement from the American Association for Public Opinion Research (AAPOR).
Update - Ben Smith reports on the sponsor of the survey of Jewish voters:
A Republican group is taking responsibility for a poll that has roiled the Jewish community by asking sharply negative questions about Senator Barack Obama.
The Republican Jewish Coalition, which is launching a campaign against Obama on behalf of Senator John McCain, sponsored the poll to "understand why Barack Obama continues to have a problem among Jewish voters," the group's executive director, Matt Brooks, told Politico.
The poll asked voters their response to negative statements about Obama, including reported praise for him from a leader of the Palestinian terror group Hamas and a friendship early in his career with a pro-Palestinian university professor. Some Jewish Democrats who received the poll - including a New Republic writer who lives in Michigan - were outraged by the poll, describing it in interviews as "ugly" and disturbing. A group that supports Obama, the Jewish Council for Education and Research http://www.jewsvote.org even staged a protest outside the Manhattan call center from which the calls originated Tuesday.
Read the full story for more details.
By Mark Blumenthal on September 16, 2008 2:38 PM | Permalink
Comments
Mark, during the 80s I worked on a phone bank that did nothing but run 'surveys' that 'tested' negative messages opposed to a ballot measure all the way through to election day. We called only long-time high-frequency voters, and it worked: The ballot measure was defeated & the talking points we 'tested' were the reasons cited by voters.
Maybe you don't engage in it, but how can you be sure that kind of 'message testing' does not take place today? It works. I saw it first hand. TIA
I think these should be illegal and subject to tort in small claims court. If there are no punishments, these techniques will continue and intensify.
Furthermore, I think it is in the interest of real pollsters to push for laws to make these illegal, because, if they don't, many respondents, like me, will simply stop talking to pollsters.
I already don't as a matter of personal policy.
Those who want to better understand how serious this is should look at:
http://www.nytimes.com/aponline/washington/AP-Push-Polls-Jewish-Voters.html?pagewanted=print
I would be interested in reading complaints by McCain supporters that they received phone calls from Obama message testers making repugnant allegations about McCain.
I think the Obama Campaign would be quite interested in such allegations as well.
Mr Blumenthal :
Your site pollster.com is a Wonder and I visit it every day. I enjoy it a lot and learn many things.
I am a Colombian living in Colombia, South America, and I can not vote in a US Election.... I am not going to go to the USA. But I plan to become an "expert" in the politics and policies of the USA and other Countries of the Western Hemisphere.
I love your maps and the clicking on states to see the graphs of trends in red and blue lines. The texts are also very informative.
The mathematical way is the key to understand elections. This is a problem in demography, psychology, the marketing of a political product, etc ...
The future of US Elections will be spellbinding because of Astonishing Demographic Changes as reported recently in US News and World Report, New York Times, etc ..
Some demographies or ethnicities will find new power at the election urns. I hope I can follow the USA future in your site.
This is FUN.
Congratulations
Vicente Duque, Colombia, South America
There's really no excuse for any organization to call people and hurl smears at them in order to test their reaction. The whole country isn't a lab, and we're not rats.
If the people doing this want to do it ethically, they should hire people or seek volunteers and do focus groups.
This is really objectionable activity regardless of how long the call might be, or how few calls are made.
David Flaum, who leads that organization is reprehensible. I hope that there will be many news stories on what a snake he is.
It is notable that Obama's wife has some relatives that are apparently Jewish:
@Scott: Great story, and it sounds like push polling, although conceivably it could be in a gray area. I thought Mark said, inter alia, that the particular surveys he describes didn't seem to be "push polls," and that all campaigns do message testing (of varying ethical standards). This doesn't seem at odds with what you said.
(Mark made the further important point that the line between "message testing" and "push polling" isn't the same as the line between ethical and unethical.)
I found this article interesting regarding the concepts of push and test polling. I found Politico's article equally interesting as they emphasized the outrage of liberal Jews over the survey statements given. Politico admits that the statements were all true--at least in part. It seems that libs are scared of Jews hearing even part of the truth about Obama's past.
Florida's Wexler is the master of expressing false outrage and attempting to marginalize what should be legitimate Jewish concerns. Wexler's credibility is waning as he attempts to explain how he can take advantage of Florida residents' tax advantages while living in Maryland and claiming his inlaws' Florida home as residence. He epitomizes unethical behavior.
I was one of the Jewish voters who received a polling call with accusatory questions about Obama. The call was quite long (more than 10 minutes at least, although I didn't time it), and asked numerous standard political and demographic questions as well as the accusatory ones. So, I agree it does not seem to fit the definition of a "push poll" that you set out, Mark.
However, push poll or not, I found this call quite disturbing. What people seem to be overlooking in this discussion is that the calls are apparently part of an effort to turn one ethnic group against another. Whether the call itself was intended to influence me, or whether it was a prelude to a negative ad campaign is not the point. Instead, what's disturbing is the attempt to deliberately turn Jews against a Black candidate based on his race.
I was asked a series of maybe 10 questions, each of them implying that Obama was pro-Palestinian and anti-Israel, or perhaps even anti-Semitic. One question I was asked, which I haven't seen reported in the articles on these calls, was whether my likelihood of voting for Obama would change if I knew that Louis Farrakhan had endorsed Obama. I'd say this question was clearly designed to make Jews feel like Obama is their enemy in a racial conflict. (I have no doubt that FARRAKHAN is an enemy to Jews.) The thing is, IF you stop to think about it, you realize that any support Louis Farrakhan expresses for Obama really tells us nothing about Obama's own opinions at all. I mean really, if a White guy and a Black guy are both running for President, would you expect Farrakhan to back the WHITE guy?
The problem is, a lot of voters aren't going to be thinking about this, especially if the commercial linking Obama to Farrakhan is playing in the background while they go get a snack or whatever. They are just going to be left with a very bad feeling about Obama, that since he's a Black guy, he must be "in league with" hostile Black guys like Farrakhan against Jews, Whites, or other ethnic groups.
That's what I see as the problem here. It's not just that this is unfair to a particular candidate, it's that it relies on race-baiting. America is full of different ethnic groups, and we just can't afford people going around trying to stir up ethnic conflict. Yet, that seems to be exactly the goal here.
Making it worse is that the issue these ads are focusing on is really a non-issue. Both parties in the US strongly support Israel. Even if Obama is secretly hiding some anti-Israel feelings somewhere, the political climate in the US will not permit him to drop Israel as out ally. It turns out that Jimmy Carter has some anti-Israel sentiments, yet he didn't harm Israel's interests during his term of office. Frankly, I would guess that the Arab-Israeli conflict is not one of the issues that is high on Obama's priority list. He probably is far more concerned with domestic issues, the wars in Iraq and Afghanistan, and aid to Africa. Even if he is somehow anti-Israel (and I don't think he is) the chance that he would squander his opportunity to make progress on these issues and go after Israel instead is pretty much nil. So, questions about his support for Israel wouldn't even be raised, if he weren't Black.
@Mark Lindeman:
You are correct. Thanks for making those points.
I should further point out that the phone bank I worked on was **not** a pollster -- though our script had us posing as one -- which would tend to support Mark B's point. Rather, it was a DC-based political consultant hired by local industry to drum up opposition.
We should not split hairs.
Push polling is not merely 'automated' many of us who are, or were, in the political consulting world have used, or have been asked to use, 'ugly' polling as a campaign device.
By my most recent tally 80% of my past candidates have personally asked me to use polling as a campaign persuasion device.
That means they have asked me to use live 'surveys' to persuade people. Their reasoning was, "I will only pay for a survey if it has a persuasion component. Ask your measurement questions first, but when you have someone on the phone make sure you persuade them."
So please, don't split hairs about push polling. Ugly polling is done with full knowledge and intentionality.
This might not be the appropriate place to post this, and if so, I'm sorry. But I wanted to point out an apparent discrepancy, in case someone hasn't noticed it yet.
Gallup today reports Obama ahead 47-45. This is from their own website:
http://www.gallup.com/poll/110446/Gallup-Daily-Obama-47-McCain-45.aspx
But on your tracking chart, you report Gallup as reporting McCain ahead, 47-45.
It's possible Gallup misreported on its website. Or it's possible a mistake was made on this end. I wouldn't hold it against either party, because hey, mistakes happen. But I'd like to know where the error is. Does Gallup show an Obama lead or a McCain lead?
Very informative blog. Nice job.
Posted on September 16, 2008 7:28 PM