I interviewedTom Guterbock, director of the Survey Research Center at the University of Virginia, on a paper he presented on tests he conducted over the last year to improve message testing poll at the annual conference of the American Association for Public Opinion Research (AAPOR).
An update on Monday's post asking whether the recorded, automated "survey" conducted by Well Point (the parent company of Anthem Blue Cross / Blue Shield) amounted to what some call "SUGGing," or selling under the guise of survey research.
The short answer, from Howard Feinberg, director of government affairs for the Marketing Research Association (MRA), is "no." The Anthem survey, which did not explicitly attempt to sell anything does not meet their definition SUGGing, a practice that MRA has taken a strong position against, along with fundraising under the guise of surveys (FRUGGing). Feinberg says that the Anthem particular call is really a form of "political advocacy under the guise of research," although it falls short of what Feinberg and most others consider "push polling."
I also received a follow-up message from Patrick Glaser, MRA's director of research standards:
I’ve spoken about the WellPoint “survey” with the MRA’s Professional Standards Chair. As planned, we will be following-up with the organization to learn more about the details of the project. In terms of MRA’s position, depending on the details of how they are presenting the activity to the “respondents,” their relationship to the respondents and how they are utilizing the information, this could potentially fall into an area where our codes do not currently provide specific guidance and/or should be discussed by the full committee.
I believe we may see these types of situations become more common, and I’m going to recommend to our Professional Standards Committee that they develop additional specific guidance about situations that are similar to this one as well as other situations that may vary a bit in their nature, but still relate to the same underlying issues.
Incidentally, our friend Desmoinesdem (who makes it a habit to answer all survey calls) blogged earlier today about what sounds like a clear cut example of fundraising under the guise of a survey ("FRUGGing"), conducted by Newt Gingrich's American Solutions political action committee. The caller identified herself as representing American Solutions, asked the respondent to participate in a "brief survey," and then depending on the responses given, ended the call by soliciting a donation.
The MRA has suggestions about what you can do about this sort of abuse of surveys and directions on how to report such calls to them directly, on their web site.
Politico'sBen Smith passes along a report from a reader who received an "automated poll" from her health insurer, Anthem Blue Cross and Blue Shield. The recorded call began with the message that "President Obama is planning to enact health care reform," and continued with these questions:
1) "Are you aware of the debates regarding health care reform?"
2) "How interested" are you in health care reform?
3) "How willing are you to get involved so we can improve our nation's health care system?"
4) "How willing are you to attend a town-hall meeting" on this subject?
5) "As Well-Point Health Care works to solve our nation's health care problems," would you like future updates?
Smith also reports that the call closed "by asking for the respondent's sex and whether they were the subscriber or spouse." He adds: "WellPoint is Anthem's parent company and is the nation's largest health insurer," with "more than 34 million subscribers."
For what it's worth, this call sounds less like a sample survey and more like the sort of massive data "harvesting" we have seen often over the last two or three years. The "would you like future updates" question suggests that the call is not about asking questions of a representative sample of a few hundred for further study, but about cheaply identifying massive numbers of customers for further follow-up via direct marketing. The last two questions (gender and "are you the subscriber?") would allow the callers to match the respondent with name on the list with reasonable accuracy.
We have seen manyrecentexamples that communicated a message while also gathering data from full populations rather than small samples. The automated, interactive-voice-response technology makes such an effort cheap and easy.
Unfortunately, these efforts put a burden on legitimate survey research. Most pollsters agree that the long term, 20-30 year decline in response rates has resulted mostly from the explosion of telemarketing calls. Many potential respondents assume that any call from a stranger is a sales call, and the sheer volume of telemarketing makes has made us all leery of any calls that begin with the tell tale sounds of an operator in a call center.
We can debate whether this sort of call qualifies as what the Market Research Association (MRA) describes as "SUGGing" - selling under the guise of research. (Update: the Marketing Research Association does not believe it does -- see "Update 2" below). At least the Well-Point calls ask permission to provide "future updates." Whether it counts as "selling" or not, we have to assume that massive data harvesting conducted under the guise of a survey makes it harder for legitimate surveys to win cooperation from potential respondents.
Update: Smith has updated his post to point to point to more details obtained by Jacob Goldstein of the Wall Street Journal's Health Blog:
The company placed three million automated calls. Of those, 142,000
connected and 66,000 people told the computer on the other end of the
line that they'd be interested in learning more, WellPoint spokeswoman
Cheryl Leamon told the Health Blog.
Insurers sometimes enlist interested beneficiaries to help sway
public opinion. "If there are members who are interested in supporting
our pos and being parts in the health care policy debate we want to
make sure that they are able to participate," Leamon said.
The Health Blog item includes a link to an online version of the "survey." Both the URL (wellpointsurvey.com) and the text characterize the questions as a "survey." On the other hand, both also identify WellPoint as the sponsor and the introduction offers no promise of confidentiality and says explicitly that "we at WellCare...need your help."
So this is definitely a massive data harvesting project. Is it ethical?
Update 2: Howard Feinberg, director of government affairs for the Marketing Research Association (MRA), emails to say, no, this particular "survey" does not fall under their definition of SUGGing. More details here.
I've lost count of howmanytimeswe'veseenithappen. A campaign testing negative messages on an internal poll gets accused of conducting a "push poll." More often than not, the opposing campaign, bristling with outrage, reaches out to a reporter who plays along. Soon we have a story quoting in which the opponents spokesperson characterizes negative message testing as "one of the most discredited and dishonorable forms of negative campaigning." Never mind that virtually all campaigns test negatives messages. Never mind that the differences between message testing and so-called push "polls" are easily discovered via Google (say here, here or here). It happens over and over.
What's unusual about today's example is that it appears on the front page of the New York Times. It was based on calls received by "several people" in New York City who picked up the phone, were told that a survey was being conducted, but "were soon asked a series of questions featuring negative information about [Representative Anthony] Weiner," one of the potential opponents of Mayor Michael Bloomberg.
The story confirms that the poll was conducted by Bloomberg. It also provides the following description of the negative information included in the poll:
The issues highlighted in the telephone calls closely echoed negative stories that have appeared in city newspapers about Mr. Weiner, and that the congressman's aides have accused opposition researchers from the Bloomberg campaign of planting.
[...]
The questions, [a respondent] recalled, accused Mr. Weiner of taking contributions from lobbyists and of securing federal funding for an organization whose members had contributed to his campaign.
Shocking. Negative campaigning may be breaking out. In New York City.
Do those facts add up to a push poll? No. Politico'sBen Smith saves me the trouble of block-quoting myself:
The story suggests the poll is a "push poll," and doesn't raise the alternative -- that it was the kind of message-testing that, those who followed the presidential campaign will recall, was often confused with push polling.
The difference is explained at length here -- "push poll" is a pollster term of art, with a technical definition -- but the main distinction is that a push poll is designed to push a sharp, memorable, negative message, typically right before an election; a message testing poll is designed to test an arsenal of attacks to see which works best, for later mail, television, or other public attacks, not to persuade the call recipient of anything.
A so-called "push poll" is not a poll at all. It's a telemarketing call made under the guise of a survey. So if the pollster makes only a few hundred calls, asks 15 to 20 minutes worth of questions, includes demographic items, those are usually clues that the intent was to conduct legitimate research for the campaign. Of course, as I've written many times, the pollster isn't off the hook just because they are testing messages. They still have an ethical obligation to tell the truth and not abuse the respondent. Unfortunately, the Times makes no effort to examine the veracity of the attacks.
If on the other hand, if the Times had evidence that the calls were very short, included just a few questions and then the negatives attacks, and were being made to many thousands of New Yorkers, there might be more to this story.
Such an effort would be important given the sort of calling that Bloomberg's pollsters, Doug Schoen and Mike Berland conducted for the Mayor's first campaign in 2001, as described in Schoen's autobiography, The Power of the Vote (pp. 281-282):
Up until this point, our work had been highly sophisticated, but not unprecedented. However, our next step proved even more radical [...] The effort that was taking shape under Mike Berland and our associate, Bradley Honan, was something new. We were now proposing to move beyond sampling and extrapolating to something new -- a census of the city, a database with information on every voter, not a sample.
"This election is going to involve over a million voters," I said. "We don't have to sample, we can do it all." By purchasing consumer information and using phone banks, we could build a profile of every swing voter that combined demographic, voting history and consumer data. We could tag every voter as a member of one of the groups and develop a dialog with them that emphasized the issues they cared about most. Every piece of mail, every phone call, would be targeted precisely on them.
Bloomberg's answer was simple and direct. "Let's do it," he said.
The theory was simple and elegant but assembling the actual database was an enormously complex endeavor: after merging voting demographic and attitudinal information, it was not uncommon to have as many as two hundred and fifty variables per voter.
So they used "phone banks" eight years ago an attempt to collect "attitudinal information" on every voter in New York City. Did these voters know the information was being collected so that advertising could be "targeted precisely on them," or did they think they were taking part in a confidential survey? If the 2001 calls were billed to respondents as the latter, then it would have amounted to what market researchers call "sugging" - or "selling under the guise of research." Not push polling, to be sure, but not entirely ethical either.
Now it would be huge leap to assume that Bloomberg's campaign is once again conducting "census" of voters, and an even bigger leap to assume that negative message testing is a part of it. However, Smith reports that Schoen is once again polling for Bloomberg. If every voter in New York City is hearing negative arguments about Bloomberg's opponents under the guise of a poll, then the "push poll" label might fit. Also, as we have learnedseveraltimes in recent years, mass political messaging campaigns are now commonly conducted under the false guise of less expensive, interactive-voice-response (IVR) surveys. Those efforts, while not a "poll" by any conventional standard, do collect the "data" provided by those who stay on the line to answer questions. So questions about the kinds of questions that Bloomberg's poll asked, and the length and mode of the survey are certainly worth asking.
However, all we know for certain in this case is that Bloomberg's pollster asked questions in a survey about negative charges that have appeared in newspaper stories. Those facts don't add up to a "push poll."
The story of the nefarious Zogby/John Ziegler "knowledge test" poll has generated some useful follow-up:
On Tuesday, Nate Silver posted the verbatim text of a contentious, occasionally profane interview with Ziegler, the sponsor of the survey.
Yesterday, Politico's Mike Allen reported that Zogby had "rejected [Ziegler's] offer to sponsor a poll to test the knowledge of people who voted for John McCain," and included an unhappy reaction from Ziegler:
"I am happy to do a poll of both Obama voters and McCain voters, with questions that I formulated and sponsored either by an objective third party or by someone on the left, in tandem with a John Ziegler on the right -- but poll questions that have my signature," Zogby said.
"I believe there was value in the poll we did," Zogby added. "I also believe it was not our finest hour. This slipped through the cracks. It came out critical only of Obama voters."
Ziegler responded: "I am shocked by John's statement that he would do another poll but not an exact duplication. What is the point of that? Not their finest hour? This a was great poll. This didn't fall through any 'cracks,' they just got scared. ... The point of the poll was for my documentary on the media's impact on voter knowledge."
This morning, The Wall Street Journal's Carl Bialik blogged a thorough review of the whole story that, as Silver points out, is "the fairest and most comprehensive summary of the issue to date" and is worth reading in full.
Over the course of the week, our own commenters took up the challenge from reader DTM, who argued that "it might be helpful if we had an alternative phrase for polls like this, ones designed not to gather information but rather to conjure up support for a preconceived claim...('agenda poll'? 'propaganda poll'? something like that)." These included "agenda poll," "prank poll," "punk poll," "faux poll," "self serving survey" and "advocacy poll" (via email). Put me down as favoring "propaganda poll."
Finally, on the issue of testing "knowledge," ABC's Gary Langer critiques the annual survey on "American Civic Literacy" from the Intercollegiate Studies Institute that claims "the majority of Americans - including elected officials - failed a test of basic knowledge about American history and economics." See Langer's post for the specifics, but he makes a point that could also apply to the Zogby/Ziegler controversy:
The reality is that the ISI itself has failed a test of basic knowledge about the definition and measurement of just what knowledge is. I've blogged about this before - see it here - but the key point is that these folks are confusing knowledge (the ability to draw on information to make considered judgments) with recall (the ability to recite disassociated facts); and then doubling down by using an inappropriate method of measurement.
Nate Silver has taken Zogby International to task for a telephone survey of 512 Obama voters that claims to "gauge their knowledge of statements and scandals associated with the presidential tickets during the campaign." Silver is right....but not right...
The summary and statement posted on the Zogby web site claims that "only 54% of Obama voters were able to answer at least half or more of the questions correctly," and more specifically that "statements linked to Republican presidential candidate John McCain and his vice-presidential running-mate Sarah Palin were far more likely to be answered correctly by Obama voters than questions about statements associated with Obama and Vice-President-Elect Joe Biden."
The survey was paid for by John Ziegler, a former talk radio host and publisher of a conservative web site. The Zogby summary quotes Ziegler claiming that "the poll really proves beyond any doubt the stunning level of malpractice on the part of the media in not educating the Obama portion of the voting populace."
The problem, as Silver points out, is that the survey does no such thing. It proves only that Obama voters surveyed were less likely to attribute to Obama or Biden a half dozen statements that were "at best debatable, yet apparently represented as factual to the respondent," such as the following:
"Which of the four [candidates] said his policies would likely bankrupt the coal industry and make energy rates skyrocket?"
"Which of the four [candidates] started his political career at the home of two former members of the Weather Underground?"
"Which of the four [candidates] quit a previous campaign because of plagiarism?"
"Which of the four [candidates] won his first election by getting opponents kicked off the ballot?"
Silver concludes -- appropriately -- that Zogby's survey appears to be less an unbiased measurement than part of "a viral marketing effort to discredit the intelligence of Obama supporters."
Zogby's defense is to deny that he conducted a "push poll" (more on that below), claiming instead that his survey represents "a legitimate effort to test the knowledge of voters who cast ballots for Barack Obama" He claims that "respondents were given a full range of responses and were not pressured or influenced to respond in one way or another." That's a little like describing the question, "when did you stop beating you wife," as fair (and as a fair test of "knowledge") by saying the husband has an opportunity to offer any date on the calendar as a response.
Zogby also claims to be a passive agent that just conducted research on behalf of a client. "The client," Zogby writes,"is free to draw his own conclusions about the research, as are bloggers and other members of society." Really? Then why does the analysis posted on Zogby's website repeatedly support "the client's" conclusions?
Zogby's survey does not amount to a "push poll" in that sense, but using the term allows him to respond -- predictably -- with a denial that "this was not a push poll." It wasn't, but that's beside the point. Describing his biased, leading questions as a legitimate test of knowledge is hugely misleading, at best.
I had been meaning to write about two new reports of unusually ugly "message testing" polls that have popped up in recent days. As usual, journalists who should know better have reached for the "push poll" label, which is not quite right. These calls do not appear to fall into that category, though as in previous cases, the surveys are pretty ugly nonetheless.
Last week, Marc Ambinder reported on calls received in Ohio and Michigan from the Opinion Access Corporation that tested negative statements about the radical views of Obama's "spiritual advisor" and presumably slanted renderings of some of Obama's votes as State Senator. A DailyKos reader named RachelMo reported receiving the same call.
This week, Jonathan Cohn details of a survey call he received apparently aimed at Jewish voters that included a laundry list of incendiary statements involving Obama and Israel. Separately, Ben Smith reported on very similar sounding calls received by Jewish voters in New Jersey and Pennsylvania.
From what I can tell, all of these calls involved live interviewers and long interviews asking a variety of questions, including standard favorable ratings, issue questions and demographic items, as well as the battery that asked for reactions to the negative statements about Barack Obama. As such, they sound more like "message testing" -- albeit very ugy message testing -- than traditional "push polls."
Some background: A true "push poll" is not a poll at all, but usually a very brief call -- the modern version is typically recorded and automated -- that isn't intended to measure anything. Instead, the purpose is to communicate a (usually) scurrilous message to as many voters as possible. Real push polls are very short and aim to reach tens, sometimes hundreds of thousands or even millions of voters. These calls are dressed up to sound like a poll in order to (a) fool the recipients into listening and (b) add credibility to the ugly messages they contain.
Message testing is different, though those differences may seem semantic to the casual observer. Virtually all campaigns ask message testing questions on their benchmark surveys. Sometimes the messages tested are positive, sometimes negative. Some pollsters will repeat their vote preference question after testing messages, because they want to see whether their message will change opinion and, if so, with what voters. In that context, they are interested in how much they can "push" opinions, but as market research for paid ads, direct mail and the campaign messages.
Most of the time the messages tested -- positive and negative -- will tend to mirror the heated, one-sided rhetoric that we hear from candidates and their campaigns every day. Sometimes, however, the messages are extreme and offensive to those on the receiving end of the call, as was the case with the two latest poll stories.
The brain-dead way to approach these stories is to argue over whether the calls amount to a "push poll." As a campaign pollster, I helped design hundreds of surveys with similar tests of messages. So trust me when I say that all campaigns -- including the Obama campaign -- test positive and negative messages in their surveys. As I've written manytimesbefore, conducting a message testing poll does not absolve the pollster and the campaign from ethical obligations. The issue is not whether the pollster is trying to "push" the opinions, but whether they are telling the truth and treating their respondents with fairness and respect.
They way I wish reporters would approach these stories is to focus less on the "is-this-a-push-poll" angle and more on evaluating and debunking the charges they include.
See more of our past coverage of push polls and message testing, Stu Rothenberg's must read on the subject and the statement from the American Association for Public Opinion Research (AAPOR).
Update - Ben Smith reports on the sponsor of the survey of Jewish voters:
A Republican group is taking responsibility for a poll that has roiled the Jewish community by asking sharply negative questions about Senator Barack Obama.
The Republican Jewish Coalition, which is launching a campaign against Obama on behalf of Senator John McCain, sponsored the poll to "understand why Barack Obama continues to have a problem among Jewish voters," the group's executive director, Matt Brooks, told Politico.
The poll asked voters their response to negative statements about Obama, including reported praise for him from a leader of the Palestinian terror group Hamas and a friendship early in his career with a pro-Palestinian university professor. Some Jewish Democrats who received the poll - including a New Republic writer who lives in Michigan - were outraged by the poll, describing it in interviews as "ugly" and disturbing. A group that supports Obama, the Jewish Council for Education and Research http://www.jewsvote.org even staged a protest outside the Manhattan call center from which the calls originated Tuesday.
Late last week, a North Carolina musician named David LaMotte received a survey call from Garin-Hart-Yang, the firm of Clinton pollster Geoff Garin. The call, as he reported to HuffingtonPost blogger and DailyKos diarist Paul Loeb, "started out normal enough" but soon "turned to long Hillary-praising and Barack bashing policy statements" with response options that asked him to evaluate each statement. At the end of the call, they asked, "now based on everything we've discussed, who would you vote for?" LaMotte used his telephone answering machine to record the latter half of the call, and as a result was the transcript that Loeb posted at DailyKos and later as streaming audio posted by Loeb, Politico's Ben Smith and ABC's Jake Tapper.
Not surprisingly, much of the commentary about this call focuses on whether the Garin survey meets the classic definition of a "push poll." It does not, at least as far as I can tell.
The call in question was long, included dozens of question that seemed "normal enough" to LaMotte and, as he confirmed to me via email, concluded with a set of demographic items that LaMotte deleted from the audio recording in order to protect his own privacy. This call has none of the hallmarks of the classic, so-called "push poll" intended only to spread a negative message under the false guise of a survey.
It was, rather, a "message testing" survey, albeit one that tested a highly negative and -- to many -- objectionable message. It was not measuring "public opinion" as it exists now but rather voter reactions to a series of positive statements about Hillary Clinton and negative attacks directed at Barack Obama. Garin asked respondents to react to each statement, and subsequently asked a second vote question ("Now based on everything we've discussed, who would you vote for?"), in order to identify the most effective attack and the voters most likely to be swayed by it.
Like it or not, this sort of testing is common in most campaigns, and almost none of the results ever see the light of day. Full disclosure: As a campaign pollster, I helped design hundreds of surveys with similar tests of messages. (I have writtenpreviously about the differences between message testing and "push polls,' see also the commentary by Roll Call's Stu Rothenberg and the recent statement on "push polls" and message testing by the American Association for Public Opinion Research-AAPOR).
Of course, simply labeling this survey as "message testing" does not absolve the pollster of ethical constraints. The pollster still has an obligation to tell the truth and treat respondents with fairness and respect. Did this survey do that? LaMotte's audio has the interviewer reading five statements that he describes as "criticisms that opponents might make about Barack Obama." After each of the statements below, the interviewer asks "if they would give you very major doubts, some doubts or no real doubts about supporting Obama."
At a time when we need leaders who are clear, strong and decisive, Obama has been inconsistent, saying he would remove all troops, but then indicating that he might not, and pledging to renegotiate NAFTA, but then sending signals that he would not actually do so as president.
He supported George W. Bush's 2005 energy bill which payed six billion dollars in subsidies to the oil and gas industry, nine billion dollars in subsidies to the coal industry and twelve billion dollars in subsidies to the nuclear power industry. It was called 'a piñata of perks' and 'the best energy bill corporations could buy.
He leads the committee with oversight on Afghanistan but failed to hold a single committee meeting or hearing on Al Qaeda in Afghanistan or anything else.
He sided with the credit card companies voting against the bill that would cap interest rates at 30 percent.
While he talks about universal health care he has failed to make the hard choices that would truly get us to universal coverage and lower health care costs for all. His plan would leave 15 million Americans uninsured.
Let's stipulate up-front that the Obama camp vigorously contests these arguments, with some support from journalists. While by no means a complete listing, here are links to reports that provide more context on the NAFTA, energy, credit card, health care and Afghanistan issues. Readers are encouraged to add more in the comments, if warranted.
However, here is the non-rhetorical question that interests me most: How much do these statements differ from those included in Clinton mailers on NAFTA, the energy bill, the credit card bill, health care or Hillary Clinton's statements on the stump about the Afghanistan oversight committee? And if they are essentially the same, why would testing these assertions in the context of a survey be any more or less objectionable than making the same assertions in a debate, a speech, a television ad or a campaign mailer?
[The original version of this post included some extraneous verbiage in the third paragraph that I've cleaned up]
Andrew Malcolm of the Los Angeles Timesreports evidence of a pro-Clinton "push poll" in California, or as he defines it, "malicious political virus that is designed not to elicit answers but to spread positive information about one candidate and negative information about all others under the guise of an honest poll."
His definition is right, but does the call in question meet it? Malcolm's source, a former local television news director named Ed Coghlan, describes a call from "a pollster who wanted to ask registered independents like Coghlan a few questions about the presidential race." The survey tested reactions to statements about Hillary Clinton and negative statements about Barack Obama, John Edwards and John McCain:
Coghlan said he was offended by such underhanded tactics and knew he was going to get out a warning about this dirty trick, but he said he played along for the full 20-minute "poll."
That last bit of information tells me that this call was almost certainly a message testing survey, and not a so-called "push poll." California has over 15 million registered voters, and roughly three million of those are independents. If "someone" was paying "to spread this material phone call by phone call among independent voters," would they really spend 20 minutes on the telephone with each one?
Not likely.
The call that Coghlan describes sounds more like a message testing survey that included many negative messages about Clinton's opponents. In other words, someone called a random sample of voters with the intent to "elicit answers," or more specifically reactions, to negative messages that the Clinton campaign or an allied group considered airing in California.
Negative campaign messages may be offensive, unfair or untrue, and it would certainly be reasonable to question the Clinton campaign on the fairness or truthfulness of the messages tested in this call. Legitimate message testing surveys sometimes cross ethical lines, especially when they raise explosive topics that candidates are unwilling to discuss openly (see the controversy over the calls in New Hampshire and Iowa that tested negative messages about the Mitt Romney's religion).
In this case, however, the only specific negative message that Coughlin reports is the attack on Barack Obama for his "present" votes in the Illinois legislature. Both Clinton and Edwards raised that issue openly in the South Carolina debate.
So far, at least, Malcolm's claim to have uncovered a "malicious political virus" operating "under the guise of an honest poll" is not supported by the facts reported.
For further reading: We have discussed the distinction between so-called "push polls" and message testing many times. Most relevant are my comments on the distinction between "push polls" and message testing here and here a well as those by Stu Rothenberg, Republican pollster Neil Newhouse and the statement from the American Association for Public Opinion Research (AAPOR).
An update on yesterday's post on the ongoing investigation by the New Hampshire Attorney General into a message testing survey allegedly conducted by Moore Information. The article I linked to asserted that Moore had "decided to cooperate" with the investigation and "agreed to turn over business records" in connection with the case. I inferred that this might mean disclosure of the identity of their client, but either the article or my inference (or both) were incorrect.
Bob Moore, president of the firm, left a comment this morning on my previous post challenging the facts of the AHN story:
The story is not accurate. Moore Information has not agreed to appear as a witness or provide documents. The subpoena was issued over our objections reserving our rights to further challenge the subpoena.
Moore Information also emailed a statement indicating that while they have provided "cooperation," they continue to challenge the applicability of the New Hampshire "push polling" law to their work:
Since this was not a push poll and since relevant statutes do not apply to the Presidential Primary Election, we believe it is appropriate to maintain the confidentiality of our client, a usual practice among opinion research firms.
See the full text of the statement after the jump.
If you were paying any attention to political news over the last few day, you have no doubt heard of Common Sense Issues, the group that has, according to the New York Times ,
begun making what it said were a million calls to households in South Carolina telling voters, according to one of the calls, that [John] McCain has "voted to use unborn babies in medical research."
"We hope to call 546,000 households in Nevada on behalf of Huckabee," said Patrick Davis, the executive director of Common Sense Issues, according to the Las Vegas Review-Journal.
Want to hear one of the calls? Here, via Ambinder, is audio (and video) of one of their calls as captured by a recipient:
Is this a so-called "push poll," an attempt to communicate an negative message under the false guise of a survey? Of course it is. But that's not the way Davis sees it, according to the Washington Post:
[Davis] questioned why McCain is characterizing the phone drive as an attempt to engage in push polling...Davis said the 45-second calls use a special technology that provides a different automated message, depending on how the recipient answers questions.
Moreover, Davis said, "A strict push poll is delivering not-truthful information. Everything we say is factual and backed up.
The Review-Journal adds:
Davis said the calls made by his group should not be called push polls because questions are asked of those called.
"A human voice is recorded asking the questions," he said. "You respond with your voice. How you respond dictates the next question. We are gathering information.
What a crock. The recipients that receive these calls are told they are participating in a survey, not a promotional message. Davis may have the high tech cover of asking questions, but the clear intent is to communicate negative messages. Check their web site's About Page. The expressed purpose of Common Sense Values is "educating and informing citizens in an in-depth manner about public policy issues." They say nothing about gathering data or measuring public opinion. They dress up their calls as "surveys" to add false credibility. Would the recipients stay on the phone if told they were about to be "educated and informed?" "Common sense" tells you they would not. The guise of a survey is a sham.
If you don't believe me, ask my colleague Nancy Mathiowetz, president of the American Association for Public Opinion Research (AAPOR):
Asking questions does not make it a survey. These calls are clearly a fraud that harm the survey research profession.
Interests disclosed: I am able to report a reaction from Mathiowetz because I am spending the day in a meeting fulfilling my duties as a member of AAPOR's Executive Council (so blogging will be light). AAPOR has a full statement condemning the so-called "push-polls" which groups like Common Sense Values conduct, and Mathiowetz devoted a HuffingtonPost blog to the subject a few weeks ago.
We may soon know** the identify of the client that hired Moore information to conduct a message testing survey that included negative questions about Mitt Romney and the Mormon religion, according to a report from AHN:
Concord, NH (AHN) - A pollster in New Hampshire has decided to cooperate with the state's attorney general in connection with allegations that a survey it conducted in November 2007 was biased against former Massachusetts Gov. Mitt Romney and Sen. John McCain (R-AZ).
Moore Information agreed to turn over business records to Attorney General Kelly A. Ayotte on Thursday, the same day a court hearing on the case was scheduled, according to the Concord Monitor.
The company was previously subpoenaed by Ayotte after it refused to disclose who had hired it to conduct what were allegedly push polls that violated state law.
The records will be examined during a closed grand jury proceeding in Carroll County. The jury will then decide if charges will be made against the company.
The report does not make clear whether the New Hampshire Attorney General can or will make public the identity of Moore's client.
**UPDATE: Either the facts of the AHN story, or my inferences from it were incorrect. Moore president Bob Moore left the following comment below:
The story is not accurate. Moore Information has not agreed to appear as a witness or provide documents. The subpoena was issued over our objections reserving our rights to further challenge the subpoena.
Moore information also sent a full statement that I have posted here.
Washington Post polling director Jon Cohen reported an easily overlooked but important statistic yesterday, especially to anyone thinking about the reliability of the last round of Iowa polls. Using the Iowatables here at pollster.com, he determined that public polls in Iowa this year have interviewed nearly 80,000 "likely caucus goer" respondents:
As a ratio of voters polled to expected turnout, this must be something of a record. (In 2004 about 120,000 people participated in the Democratic caucuses, and in 2000 about 90,000 in the GOP contest.)
And it's not just the public pollsters calling. Campaigns have been known to set up a phone bank or two to gauge opinion, solicit support and cajole voters to actually show up and spend hours caucusing in the middle of winter.
A month-and-a-half ago, already deep into the "silly season" but well before the final stretch, eight in 10 likely Democratic caucus goers and nearly six in 10 on the GOP side said they'd been called on the telephone by at least one of the campaigns. And Pew reported the pervasive use of robo-calls (though most Iowans who get such automated calls about the campaign said they usually hang up).
I can add two confirming anecdotes. The first comes from a comment left by "Randy Iowa" here at Pollster just last night:
Is there a Do Not Call list that i can get on? I have received a survey call everyday this week and at least one candidate has called everyday as well.
I emailed Randy, and sure enough, he is an Iowa voter. He says that "80%" of the calls he received were automated. Interestingly, he is also a non-affiliated voter (not registered with a party) registered independent who has never participated in a caucus (though has "voted Republican my entire life"). (By the way, the short answer to Randy is no. Pollsters and political campaigns are exempt from the federal do not call restrictions, though at least one group is trying to change that).
I wonder how many calls those identified as past caucus goers are getting? Here is one possible answer in he form of an email I received about an hour ago from a "help desk" operator at a major residential telephone company. He apparently assumed (mistakenly) that Pollster.com conducts surveys:
Subject: Please stop calling this customer
This customer is getting upwards of 20 calls a day from automated poll services, she lives in Iowa and her phone number is 563-[redacted]. Please stop calling her.
Not surprisingly, the recipient of the calls lives near Davenport Iowa.
Aside from spectacle of the sheer volume of "poll" calls, we might want to think about what all that calling is doing the the response rates the real pollsters are getting. And if pollsters are having a harder time getting voters to respond this week, are those suddenly reluctant voters skewing the results? We may never know, of course, but if nothing else, I would be very nervous were I using an automated (IVR) methodology to collect survey data in Iowa right now. More important: I wonder how many many Iowans have been ignoring their ringing phones altogether the last few days?
Kathy Frankovic reviews data showing Democrats are "far more energized" and attentive to the presidential campaign than Republicans.
Frank Newport suggests the "Huckabee boomlet" may have "leveled off" nationally, and ponders the 62% of Republicans who say they are "afraid" of a Hillary Clinton presidency.
Chris Cillizza reports on response to Hillary Clinton from a Washington Post post-debate focus group in Iowa.
Mark Mellman breaks down the strategies of the Democratic presidential candidates.
David Hill thinks Hillary Clinton "may have blown her chances" by unleashing Bill Clinton in recent weeks.
Gary Langer argues that the Mitchell Report on steroids in baseball misconstrued data on teen steroid use.
Carl Bialik, in his Numbers Guy column and blog, looks at shortcomings in the "top searches" that the top Web search companies report for 2008.
Via The Page, ABC's Brian Ross and Avi Patel have a report on more strange calls in Iowa asking mostly negative "message testing" questions about the Democratic presidential candidates. They even managed to get a tape recording. It's worth reading (and listening to) in full, but here is the lead:
Iowa Democrats are being hit with a new round of "opinion poll" calls this week that stress negative qualities of Sen. Hillary Clinton and Sen. Barack Obama and praise John Edwards as a man who "has spent his life fighting powerful interests."
The calls come from operators who say they are "out of state" and are conducting an opinion poll for a "research company."
Ross and Patel call it a classic "push poll" fraud, but that conclusion is not evident from the questions themselves. It is not clear from the story, for example, whether the calls included demographic questions or other items that are typical of a longer "message testing" survey. Either way, the mystery continues.
The verbatim questions in the ABC report bear a strong resemblance to calls reported by Politico's Ben Smith and a HuffingtonPost OfTheBus poll project respondent last week. Consider the following question transcribed verbatim from the ABC call:
Question #7: Barack Obama has taken over $12 million from the financial industry and its lobbyists. In the Senate, he was one of the only 15 Democrats who supported the financial industry by allowing predatory lenders to target (unintelligible and the poor by charging unlimited interest on their credit cards and loans. Is this very concerning, somewhat, not so concerning or not at all concerning?
Now compare it to this question reported by Ben Smith's correspondent:
Barack Obama has taken over $12 million from lobbyists and other special interests since his time in the Illinois legislature. Does Barack Obama taking money from lobbyists and special interests trouble you?
And compare again to this question reported by the OffTheBus respondent:
[Asked] If I knew that Barack Obama (pronounced incorrect! I had to instruct the interviewer how to pronounce his name!!) took over 12 million from the financial industries and voted to allow credit card companies to raise interest rates as high as they wanted to, would that affect my opinion of him?
I will yield to anyone with better Googling skills who can find a reference, but when I search on "Obama '12 million' lobbyist," I get nothing similar to the attack above other than my own blog report last week. As such, it seems reasonable to conclude that these calls are coming from the same source.
I am curious (if anyone at ABC News is reading), whether the call included any other questions during the course of the interview, either demographic questions or more standard survey questions.
Developing? We'll see.
PS: One more thing. Commenter "In Iowa" leaves this this tantalizing report on the ABC story:
I got this call a couple of days ago. Recognizing it for what it was, I asked the caller who was conducting the survey. According to them, they work for a telemarketing firm in Ohio called "Influent", and they refused to say who was paying them to do the calls. They say they work for anyone that pays them without regard to political party. They stated that they were not required by law to tell me who was financing the call.
If this comment is legitimate (and we always be careful about trusting anonymous comments), it might connect these calls to the ones Ben Smith traced last month to the same company (Influent). Those calls asked if respondents would still support John Edwards after learning that, as one respondent remembered, "he chose to continue the presidential campaign instead of staying home with his wife who has cancer."
More strange "poll" calls from Iowa. Ben Smith has posted details of a call received by a reader in Eastern Iowa of a short poll that tested three messages. Smith summarizes them as "roughly" the following:
1. While in the Senate Hillary Clinton was one of the first Democrats to support Bush tax cuts for only the wealthiest of Americans at the expense of the middle class, do you find this...
2. Barack Obama has taken over $12million from lobbyists and other special interests since his time in the Illinois legislature. Does Barack Obama taking money from lobbyists and special interests trouble you?
3. John Edwards has taken the position that all troops in Iraq will be out within one year of his being elected president, do you find this rhetoric irresponsible?
Smith ads that the caller "said she was with 'Independent Research,' and the caller ID showed a phone bank called DRS Acquisitions." DRS Acquisitions is a company that, as its web page says, specializes in "national outbound telemarketing programs." I emailed Smith, who shared the reader's full description. The call also asked first about the respondents likelihood of attending the caucuses, his vote preference and a question about the type of candidate he most favors:
1. Fighter against special interests
2. Bold fighter for middle class families
3. Committed to getting us out of Iraq
4. Fight for universal healthcare
The negative message test followed the item above. Smith's reader also reports that he interrupted the interview, and ultimately ended the call, after hearing the negative messages.
This call sounds similar to one I found among the calls reported by the first wave of respondents to the HuffingtonPost OffTheBus polling project (we are sponsors). An Edwards supporter who lives near Des Moines reported receiving a call last week. The caller insisted he was working for "an independent polling company," asked her vote preference and the certainty of her support and then asked about the following:
[Asked] If I knew that Barack Obama (pronounced incorrect! I had to instruct the interviewer how to pronounce his name!!) took over 12 million from the financial industries and voted to allow credit card companies to raise interest rates as high as they wanted to, would that affect my opinion of him? I answered yes.
If I knew that John Edwards was a liberal trial lawyer would that affect my opinion of him? I answered yes.
Do I feel that the American Dream is achievable by average Americans or do I think the system seems to be rigged to favor the rich. - I said - rigged.
One more question that I didn't get written down right away so I don't remember it exactly...it was along the line of Which would be most upsetting to me - a candidate who would roll back the Bush tax cuts, or ...2 other answers that I can't remember. Sorry!
Added that their survey ended with demographic questions on age, income and union membership.
So what are these calls about? I have no idea who is making the calls, and we will drive ourselves crazy trying to deduce the sponsor or sponsors from the facts above, or even whether Smith's call was part of the same project as the OffTheBus report. The entity or entities behind these calls are clever enough to include "negatives" on all three front-runners, making it virtually impossible to deduce much of anything about sponsorship from the questions themselves.
These calls do not seem to fit the profile of the classic "push poll" dirty trick. In retrospect, I think we can say the same of the calls Smith reported a few weeks back that he traced to a telemarketing company named Influent. That is, these do not appear to be calls masquerading as a poll that intends to measure nothing and exists only to spread a malicious rumor. The calls described above include too many questions that would be pointless as part of a fraudulent "push poll" dirty trick.
On the other hand, these calls resemble few internal campaign polls I have seen, and sound nothing like what I would expect from the internal campaign pollsters. They are far too short, and leave out many of the sorts of questions that campaign pollsters typically ask at this stage of such a well funded, closely watched campaign. And with the caucuses less than a month away, it is a bit late for true "micro-targeting." So the exact nature of these calls remains a bit of a puzzle.
Still, I do see two stories here. The first is about the blurring of the lines between traditional polling and telemarketing. Campaigns have for many years used paid call centers to conduct "voter ID" calls, that is, calls intended to identify supporters and those still undecided. Increasingly, those calls have grown to resemble polls. However, the use of such calls to test negative messages is something new in my experience.
And second, whatever the nature of these calls, I think we can also conclude something about what they portend. Political commentators, including yours truly, have been speculating about when the Democratic television advertising will turn negative in Iowa and New Hampshire. At this point, for a variety of reasons, such a turn seems unlikely. However, these calls tell us a very well targeted negative mail campaign is imminent from someone, or more likely, several someones. Those looking to cover "Act III" of the Iowa campaign would be well advised to watch the mailboxes of Iowa voters.
Meanwhile, if you have been on the receiving end of one of these calls, please email me or report it to HuffPo's OnTheBus polling project.
Republican Pollster Neil Newhouse
sends Politico a "brief pollster's guide to what constitutes a push poll and
what doesn't."
Kathy
Frankovic reviews the Romney "push poll" story and the confusion between
"push" polls and real polls, reminding us that real candidate polls sometimes
include negative information.
Jennifer Agiesta reports the highest level of economic
pessimism in 17 years, as measured by the Washington Post/ABC Consumer Comfort Index, and
recaps data on "personal
contact" with the candidates by likely Iowa Caucus goers. .
Frank Newport notices that George Bush is poised to break
Richard Nixon's record of 14 straight months of a job approval rating below
40% and sees parallels between now and 1992 in terms of Americans' "perceived
distress."
Jay
Cost checks the internals on the recent Post/ABC
Iowa poll and concludes that Hillary Clinton is relying on first time caucus
goers as much as Barack Obama.
Tom
Bevan does some apples-to-apples comparisons of the trends in Fred
Thompson's support (via Martin)
Mike
Lux reports seeing a "clear trendline" in "both public and private" polling
in Iowa that shows Barack Obama "has picked up several points, and is now at
least tied with Clinton, and maybe even up a little" (via Smith).
Dante Scala has new ad buy numbers for Giuliani,
Romney
and Paul
on New Hampshire's
WMUR.
DailyKos diarist DHinMI is not happy with John Zogby for conducting a survey commissioned by a Ron Paul supporter.
I was going to try to finish up my Disclosure Project post again today, but alas, there are more timely reports on the mysterious anti-Romney poll worth commenting on, a controversy some are now dubbing "Mormongate."
First, a summary of developments since I wrote about this story last week. A handful of conservative bloggers -- particularly Soren Dayton and Liz Mair -- have been theorizing that Romney supporters or the Romney campaign itself might be responsible for the calls. That speculation culminated in a widely cited article by the National Review's Mark Hemmingway, which argued that "evidence points" in the "general direction" of the Romney campaign as the culprit. Hemmingway's article drew a forceful response from both the Romney campaign and their pollster, Alex Gage of TargetPoint Consulting:
To set the record straight: TargetPoint Consulting has absolutely nothing to do with the calls in question. To be even clearer: TargetPoint Consulting has NEVER and will NEVER conduct a push-poll. TargetPoint is in the business of promoting Governor Romney, not manufacturing fantasy plots that involve smearing him.
Gage's denial is clear and unequivocal. However it is worth noting that nearly everyone involved, including Western Wats (the survey call center that allegedly made the calls in question) is eager to deny they conducted a "push poll." As I wrote last week, based on the reports of respondents, these calls sound more like a "message testing" poll than the classic "push poll" dirty trick (which is not a poll at all but calls that impersonate a real survey in an effort to spread a nasty message -- see my prior post). The questions worth asking, at this point, are who sponsored the calls and what did they say, exactly, about Romney's Mormonism?
Today, Dayton and Mair are asking why so many of the respondents that have come forward to report receiving the calls are either Romney supporters or paid Romney staff. The obvious explanation is that the Romney campaign directed supporters that were called to reporters. And in the latest development, TPM's Greg Sargent now reports that the Romney campaign confirms it did just that -- "referred reporters to two recipients of the calls without disclosing that the two were also on the Romney campaign payroll."
The two conservative bloggers are alleging that these ties to the Romney campaign imply something more sinister: Either a poll specifically targeted at Romney supporters in order to create a story, or perhaps an effort to get Romney staffers and supporters to lie to reporters about a non-existent survey.
For what it's worth, both allegations seem like a quite a stretch, but I want to answer one specific question raised by Liz Mair from my own perspective as a campaign pollster.
She wonders whether "including people on Romney's payroll or those publicly affiliated with the campaign in a call sample would be bad, and non-standard, practice." Not really. I cannot remember conducting any poll that attempted to exclude campaign staffers or those "publicly affiliated" with any campaign from the sample itself, as the registered voter lists typically lack any such information. Sometimes, when sampling from a list maintained by a political party, sample vendors offer the option of excluding those identified on the list as party activists or precinct captains, but that practice is far from standard.
Mair also goes on to cite a "top Republican pollster" who answers her question:
[N]ormally, people working for or associated with campaigns, and members of the media, are excluded from calling lists in the first place-- presumably because of bias that might be evident, which could raise questions about the accuracy of the result.
There may be some confusion here about what this particular pollster meant by "list." Some pollsters -- but by no means all -- begin their interview with what some describe as a "security screen." It asks if the respondent works for a political party, a campaign, a news organization, etc., with the aim of screening out such respondents from the final sample. I have always been skeptical that such screens accomplish what they aim to. My old firm typically used such screens only when a client specifically requested it. Pollsters may disagree about the merits of this procedure, but describing it as a universal or standard practice among campaign pollsters is just not accurate.
So should we be suspicious that most if not all of the respondents that have come forward are Romney supporters? I'm not sure we should. First, Romney supporters have the strongest motivation to come forward. They are likely more angry and upset and are more likely to want to report the calls to reporters or the Romney campaign. Second, as argued by Dayton and Mair, the Romney campaign has incentive to try to direct angry supporters to reporters in hope of generating a sympathetic story.
Third, we ought to think about the implications of the size of the sampled universe and the cooperation rate that pollsters are currently receiving from Iowa voters. Consider that the all time high Republican caucus turnout was little over one hundred thousand. Past caucus goers on the lists are the most active and committed Republicans in Iowa. Consider also that nearly every campaign and many different pollsters have been calling into Iowa in recent weeks, and that is on top of automated recorded calls placed by each campaign. Given that the best of surveys conducted under the best of conditions get response rates in the 20 to 30 percent range, and assuming that native campaign staffers and activists are probably the most likely to cooperate, the odds of getting a disproportionate number in the sample seems likely. The point here is simply that the odds of including a half dozen or so active Romney supporters (and even a paid staffer or two) in a sample of 600 or so Iowa Republicans do not seem terribly long to me.
So bloggers will speculate and dig further, as we always do, but I am not convinced from the facts before us allow the conclusion that pollster behind this survey intended to contact only Romney supporters.
Update: I had not seen it, but NRO's Mark Hemingway responded to criticism of his initial article today. His post includes a series of questions that TargetPoint refuses to answer beyond their blanket denial: "we had nothing to do with these calls." (via Sullivan).
The story of the anti-Romney poll calls into Iowa and New
Hampshire that I wrote
about yesterday gets stranger and stranger. Here is the lead of the story reported
last night by the Chicago Tribune's
Jill Zuckman:
The GOP presidential campaigns of
Mitt Romney and John McCain-rocked in different ways by a highly negative
"push poll" targeting Romney's Mormon faith-demanded Friday that the
New Hampshire attorney general investigate who is behind the tactic. The
attorney general's office said it was investigating the phone calls.
Again with feeling: This particular set of calls sounds more
like an ethically questionable "message testing" survey than a classic "push
poll." See my post
from yesterday for more details on that issue or the clarification
released last night by the American Association for Public Opinion Research
(AAPOR - full disclosure: I serve on AAPOR's executive council).
An interesting twist to the story, according to Zuckman's
story, is that a "New Hampshire law requires all political ads-including
phone calls-to identify the candidate behind the effort, or at least the
candidate who is being supported."
I went looking for more details about the questions asked on
the calls, and the most detailed report comes from State Representative Ralph
Watts, a Republican from Adel, Iowa. He taped a radio
interview with Radio Iowa that you can listen to online.
Here is the way he describes the interview (my transcription):
It started out like a lot of
telephone polls do these days. They wanted to know if I was a caucus goer, and
whether I was a regular voter and all that usual stuff. And then it progressed
into questions about Mitt Romney, and specifically about the Mormon Church.
The first one, I guess, was
innocent enough. It asked a question whether I would be more or less likely to
vote for Mitt Romney because he's Mormon. Well, I guess that's a fair question,
but not necessarily a pertinent question. And then it went on to talk about the
philosophy of the Mormon Church. Would I be more or less likely to vote for
Mitt Romney based on some of the tenants of the Mormon Church?
[snip]
This telephone interview went on
for about 20 minutes. The last half of it were questions directed, they were in
a more positive light and they were directed toward John McCain. They asked a
question, what if I knew that McCain had some 330-some carrier landings and was
a Navy pilot would that make me more or less likely to vote for him. If I knew
that John McCain were a prisoner of war in Vietnam would it make me more or
less likely to vote for him. Then there was a whole series of questions about
John McCain that were very favorable questions about John McCain. It would have
led one to believe that John McCain were behind the poll, but that would have been too obvious.
And I've done some checking myself
and [with] some people, and I'm convinced that John McCain had nothing to do
with it. Who actually did it, there you don't know.
What Watts describes starts
out with typical political survey questions, then shifts to a long series of
negative arguments about Mitt Romney followed by a long series of positive
arguments about John McCain. The length of the interview and type of questions
is indicative of a "message testing" survey. Ordinarily, that pattern would
suggest a survey conducted by someone supportive of McCain looking for the best
ways to promote their candidate and to most effectively tear down Romney. However,
between the red-hot spotlight of presidential politics and the incendiary
nature of questions about Romney's religion, there is nothing ordinary about
this survey.
It is tempting to try to use the facts reported by Watts and
other respondents to logically deduce the identity of the sponsor of the calls.
But readers need to remember two things about contemporary "push poll" stories:
First, respondent memories are often imperfect. They will
often exaggerate some details and omit others. Consider that in a 20-minute
interview, a pollster can typically ask 60 to 80 questions. In the description
above, however, Representative Watts specifically recalls just a half dozen or
so questions. My point here is not to challenge his story, only to suggest that
the reports we have are so far cover only the most memorable details. We may be
missing some useful context.
Second, and probably most important, keep in mind that
accusations of "push polling" have become a fact of life for campaign
pollsters. Since virtually all campaigns in both parties now conduct "message
testing" surveys, and since most reporters reflexively (and erroneously)
describe any report of a negative question on a survey as evidence of "push
polling," pollsters have grown accustomed to being so accused. Unlike the calls
involving the Democrats I wrote
about earlier in the week, these calls have all the hallmarks of a
professional survey, including the length of the questionnaire and the use of a
well-regarded call center. So given the intense media spotlight on Iowa and New
Hampshire and the explosive nature of questions about Romney's Mormonism, my
guess is that the pollster that designed this survey assumed the calls would
lead to a "push poll" story. Perhaps that assumption is a part of their
strategy.
And that's what makes it impossible to try to deduce from
the available facts the campaign or interest that was behind the calls. As
Representative Watts says, the positive questions about McCain are almost "too
obvious" as a ploy intended to implicate McCain is the sponsor.
So who is behind these calls? I haven't a clue, but the
story gets stranger and stranger.
Update: Jonathan Martin has the latest on this story here and here.
Another day, another "push poll" story. This time, Politico's
Jonathan Martin reports
on an "apparent push poll" in Iowa
involving a "research firm" that "called Iowa Republicans this week
praising John McCain and criticizing Mitt Romney and his Mormon faith." AP's Phillip
Elliot traced
the calls to:
Western Wats, a Utah-based
company, placed the calls that initially sound like a poll but then pose
questions that cast Romney in a harsh light, according to those who received
the calls.
Elliot then leaps to the same quick shortcut that tempts all
to many reporters:
In politics, this type of phone
surveying is called "push polling" - contacting potential voters and
asking questions intended to plant a message in voters' minds, usually
negative, rather than gauging peoples' attitudes.
Noit'snot. The information
described in the reports by Martin and Elliot sounds more like a form of
message testing done by a real pollster - not the classic "push poll" dirty
trick, although that distinction does not absolve the pollster from ethical
responsibility for the content of their questions.
In writing about
this issue I have tried to distinguish between the classic so-called "push
poll," which is not a poll at all. It has no "sample" (in any statistical
sense), no data collected, no analysis. It just amounts to someone making phone
calls to spread a nasty rumor under the guise of a survey.
What confuses everyone is that campaign pollsters routinely
conduct surveys that test campaign messages and try to simulate the dialogue of
a real campaign. That message testing can often involve negative information. As
Guiliani pollster Ed Goeas told John Martin:
"When you're doing a research call
you ask positive and negative questions on [your own candidate] and positive
and negative questions on [your opponents]," he said. "You're trying
to war-game."
In this case, the calls apparently came from a survey call
center known as Western Wats that acts as a vendor for many legitimate pollsters
and survey researchers. The calls reported were part of a longer interview. Elliot
included this account from one respondent:
The first 15 or 20 questions were
general questions about the leading candidates," she said. "Then he
started asking me very, very negatively phrased questions about Romney. The
first one was would you have a more favorable, less favorable, blah, blah,
blah, impression of Mitt Romney if you knew that his five sons
had never served in the military and that he considered working on a
presidential campaign as public service or some such question.
Based on those descriptions, these calls sounds like some
sort of "message testing." But tossing aside the "push poll" label does not absolve
the pollster of ethical responsibility. At a minimum, as the statement by the
American Association for Public Opinion Research (AAPOR) puts it:
[Message testing] surveys should be
judged by the same ethical standards as any other poll of the public: Do they
include any false or misleading statements? Do they treat the respondent with
fairness and respect?
The respondents quoted in the two news stories were certainly
disturbed and angered by the questions they heard. Consider also the details
these respondents remembered. Martin passes on the report of one respondent:
"Statements were on baptizing
the dead, the Book of Mormon being on the level of the Bible, and one about
equating it to a cult," said the Iowan, deeming them "common
criticisms of Mormonism."
AP's Elliot added:
Among the questions was whether a
resident knew that Romney was a Mormon, that he received military deferments
when he served as a Mormon missionary in France, that his five sons did not
serve in the military, that Romney's faith did not accept blacks as bishops
into the 1970s and that Mormons believe the Book
of Mormon is superior to the Bible.
One thing we can say without jumping to any conclusions
about who may be responsible: No campaign has made these sorts of statements or
attacks openly, and the organization that paid Western
Watts to make the calls has so far been unwilling to take
responsibility for the survey. So even if these calls were part of real survey,
even if the information was narrowly factual in some sense, the refusal of the sponsors
to accept responsibility for testing it speaks volumes about the ethics of the
test itself.
Yes, campaigns (and independent groups) have the right to
privately consider strategies they ultimately decide not to pursue. But when the
market research for those potential strategies touches hundreds (or thousands) of
volunteer respondents with a message that deeply offends, and when the
organizations that sponsor the research hide behind a cloak of secrecy,
something is very wrong regardless of the label we use to describe it.
By now many of you know about a new outbreak of nasty phone
calls in Iowa.
I have to say that this episode truly puzzles me. Politico's Ben Smith reported
the key details yesterday:
Two bloggers today reported receiving calls last
night from a pollster testing whether John Edwards' failure to drop out to take
care of his ailing wife could damage his campaign.
The pollster asked whether
"desmoinesdem," a well-regarded liberal Iowa blogger, would not support Edwards
because "he chose to continue the presidential campaign instead of staying
home with his wife who has cancer," the blogger reported. A blogger
on John Edwards campaign website, doridc, shared a similar
recollection.
The two blogger/respondents also said the "survey" included
negative statements about both John Edwards and Hillary Clinton, which led both
to believe that the calls came from someone associated with the Obama campaign.
My initial reaction was that these calls were part of some
sort of message testing rather than a so-called "push poll," and that a real
message testing poll could have involved any
of the Democratic campaigns. As regular readers know, a "push poll" is not a
poll at all but political telemarketing -- usually an effort to spread a nasty
negative message -- under the false guise of a legitimate poll. However,
campaigns sometimes use real surveys to test potential negative messages for
advertising. I've written more about the differences here; see also Stu
Rothenberg's piece and the AAPOR statement on "push
polls."
If this were a real "message test," the absence of
statements about Obama or any of the other Democratic candidates does not
necessarily implicate either Obama as the sponsor (or Dodd, Richardson or
Biden; Marc
Ambinder made a similar point yesterday). As a campaign pollster, I always
included "negatives" on my own client whenever I tested potential attacks on
the opposition. The idea was to try to keep some semblance of fairness and
balance in the questionnaire (so respondents would not immediately hang up in
anger) and to try to simulate the effects of the likely exchange of attacks
that would occur if the campaign "went negative." For what it's worth, I know
that Harrison Hickman, the Edwards pollster for whom I once worked, has always
taken the same approach.
Ben Smith, who has been all over this story for the last 24
hours, found
a real call center that uses the name "Central Research" in New York and
another Central Research in Arkansas, and speculated about projects they have
done in the past. However, he subsequently reached the people that run those
companies and each flatly
denied any involvement. He also reports that each of the Democratic
campaigns - Obama, Clinton, Edwards, Biden, Dodd and Richardson - "all quickly
denied this was their poll."
Puzzled, I went back and re-read the reports of the first two
blogger/respondents (as well as another respondent account Smith reported
this morning). All tell essentially the same story:
Both
report being asked first about their voter preference in the Democratic
caucuses, about the strength of their support for their first choice and
about their second choice.
Both
report being asked just two more questions: A three-way forced choice
question asking them to choose from among three statements as the
"most important reason not to support Clinton." They were then asked a
similar question with two negative statements about Edwards.
And
then, both bloggers report (or at least imply) that the interview ends abruptly..
"End of survey" says desmoinesdem. "There
were no questions about what issues are you interested in," writes dorisdem, "or
even how likely are you to caucus." Neither mentions being asked demographic
questions (what is your age, race, etc.), although is it possible both simply left
that part out.
If the "poll" included just the five questions above, it fits the profile of
the a real so-called push poll,
again, not a poll at all but a negative "advocacy call" masquerading as a
legitimate survey.
The
"negative statements" are also strange, and not just because of the outrageous
and incendiary reference to Elizabeth Edwards' fight against cancer. The two
bloggers report essentially the same statements. This is dorisdem's memory:
[They ask] why do you think Hillary Clinton is a
weak candidate and gives 3 choices. A) Is a weak general election candidate.
B)Is too dependent on lobbyist money. C) Won't bring change.
Then why do [you] think John Edwards is a weak
candidate with 2 choices A) a weak general election candidate because his
positions are too liberal B) He should be home with his wife who has cancer.
These
are just not the sort of statements that I can imagine any of the campaigns
wanting to "test" in this form at this stage of the campaign as potential
fodder for television or direct mail advertising. Think about the ways the
campaigns are criticizing each other now in speeches, online videos and
debates. The statements in the calls make no reference to votes on Iraq, Iran,
trade policy or double-talk
in regards to Senator Clinton; nothing about hypocrisy, being too negative or
"piling on" in regards to Senator Edwards.
These
oddly constructed questions look mostly to me like a clumsy attempt to dress up
as a "poll" the beyond-the-pale reference to Elizabeth Edwards' illness.
At
very least, I find it utterly inconceivable that Harrison Hickman or the
Edwards campaign had any connection to a five-question survey of this sort, and
extremely implausible that it was part of any real poll conducted by anyone
else.
So
who would be doing this?
The
bottom line is that I have no idea who is behind it, but we ought to consider
another scenario as at least as plausible as the notion that this came from one
of the Democratic campaigns. It is also possible that this was the work of some
independent group with Republican ties that sees some value in gathering crude
information about the Democratic race while fomenting ill-will and infighting
in the Democratic ranks. If that was the goal, you need only read the comments
under the posts of the blogger/respondents I linked to above to see evidence of
just that happening.
But
again, we really have no idea. It could be anyone, and we'll probably never
know.
PS:
Just to help clarify the record, blogger/respondent doridc makes
two statements that are not quite right. First:
In polls from reputable sources they never ask for specific
voters as was the case here.
Not
true. Most surveys that sample from registered voter lists, including just
about all of the campaigns, ask for the name on the list when they call. They
can randomly select an individual in the household (from the list). Asking for
a specific person by name allows them to match up the answers to actual vote
history on the list for analytical purposes.
She
continues:
Also they used my voter id as listed in the Iowa Democratic
Party database because my full name does not fit, it is missing the last 3
letters. So it was a candidate driven call.
Not
necessarily. As I understand it -- and I have this information from individuals
with firsthand knowledge of the process -- the list that the Iowa Democratic
Party sells to campaigns is built by appending caucus "vote history" data that
they collect to the registered voter list provided by the Iowa Secretary of
State. I may be wrong on thus, but I assume that if doridc's name is truncated
in the file, that truncation occurred on the Secretary of State's list that is
available to anyone.
UPDATE: Ben
Smith links to this post along with confirmation oF one point above:
After talking to Blumenthal today, I went back to one of the
respondents, "desmoinesdem," to ask her about some of these details: Was
she asked about whether she plans to vote, or about her age or party
affiliation. She wasn't.
Desmoinesdem has also updated her original post with another interesting observation:
In the comments, yitbos96bb suggested a possibility that hadn't occurred to me. The pollster may be testing negative messages against Hillary (the front-runner) and whomever the respondent supports. So doridc and I got the negative messages about Edwards, but perhaps if we had named a different candidate as our first choice, we would have gotten the questions about Hillary and our first choice. A Republican group paying for a poll like this might be testing to see what kind of messages would work best against Hillary and whomever Democratic respondents lean towards.
Yesterday, the Iowa Independent news website ran a story about
a Democrat from Iowa city who says he participated
in a long political survey that tested reactions to positive statements about
Hillary Clinton and negative statements about John Edwards' "$400 haircut" and
Barack Obama's votes "to fund" the Iraq war. Politico's Ben Smith linked
to that story, as well as a recent DailyKos diary
about a similar call received by a New Hampshire Democrat that mentioned the
recent unflattering article about Edwards in the New York Times Magazine. TPMCafe's Greg Sargent located
another respondent from Iowa
and noted that all three said the call came from a firm called "PSA
Interviewing," the telephone call center of the firm of Clinton Pollster, Mark
Penn.
This is not the first such story to involve surveys testing
negative messages about Clinton's
opponents originating from "PSA Interviewing." Similarreports
a few months made it into the profile of Penn by The Nation's Ari Berman Melber, including a
response from Penn that "the charges were false and that ‘this firm conducts
standard political and market research polls...and does not do push polling.'"
Two reactions:
1) No, Ana,
and no, Taegan,
it is not a "push poll." TPMCafe commenter "slcathena" gets it exactly
right:
It's not a push poll. It's just
this side of a fine line between message testing, and a push poll, but it's not
a push poll. Now, were it a 30 second to 1 minute call with just negatives,
going to tens of thousands of people (ie, not a standard 300-1000 sample size)
THAT would be a push poll.
Remember, a "push poll" is not a poll at all but an effort
to communicate a message under the guise
of legitimate research (more here and here).
And let's give due credit to Greg Sargent, Ben Smith and the Iowa Independent's
Chase Martyn for avoiding the "push poll" label altogether.
2) Even if only
"message testing," the story does not end there. Pollsters still have an
ethical obligation to tell the truth to respondents, and this incident raises
some interesting questions about whether campaigns should be willing to take
ownership of the messages they allow pollsters to test.
In this case, no one seems to be questioning the truthfulness
of the messages tested (although we have not seen the verbatim text). What
seems more at issue is whether these sorts of negative attacks are appropriate,
even if technically true.
Consider the context: Message testing" is ubiquitous in
campaigns. Virtually every campaign that hires a pollster will conduct surveys
that test messages, and most will test negative messages about their opponents.
In my career as a campaign pollster, I wrote hundreds of surveys that did exactly
that. And I can testify that campaigns frequently test messages they opt out of
using in the campaign. At this stage, they are keeping all options open.
Campaigns also consider the benchmark message testing survey one of the most
closely held documents in the campaign and are hugely reluctant to discussing
details with reporters.
What I find fascinating is the way the Internet is forcing a
change in that culture. Ten or twenty years ago, if a voter participated in a
"message testing" poll, they might have the same angry reaction as the
respondents quoted in the stories above. They might mention their experience to
a friend or colleague, but few bothered to call a reporter. Now, however, if
you call 600 or 1000 voters, the odds are good that a handful will know how to
leave a comment on a blog, and rather than ask friends or family, they will
turn to thousands of readers of, say, DailyKos and ask, "what the heck was
that?" And given the nature of the blogosphere, one comment will beget another,
and these various testimonials will quickly get into the hands of political
reporters.
All too often in the not so distant past, campaign
consultants operated under the illusion that they could test the "family
jewels" of a campaign in secrecy. Now, the reality is that if you put it on a
questionnaire, especially in the context of a high profile campaign, it stands
a good chance of being discussed somewhere on the Internet and found out by the
political press. As such, campaigns will need to reconsider their willingness
to take responsibility for the messages they test.
Last year, I argued
that message testing polls "deserve the same level of scrutiny as any charge or
statement made in the political realm." I think that works in both directions. We
ought not holler "push poll" whenever someone tests a negative message on a
legitimate survey, implying that the research is somehow more ethically
questionable than running the same message in a television add. Similarly, we
ought not exempt the testing of those messages from criticism simply because it
is research.
In my last few years as a campaign pollster, I tried to give
my clients the same advice: Don't put anything in a message testing
questionnaire that you are not willing to publicly defend. If the Clinton campaign is
willing to test the negative messages alleged above, they ought to be willing
to take ownership of those messages and the tactics they imply. If not, then we
are all left to draw our own conclusions.
Before I leave the topic of "push polls"
and the indiscriminate way reporters and pundits use that label, I want to
mention a prominent example that appeared earlier this month. Conservative
commentator Robert Novak led off his March 1 column
with this reference to a survey of Republicans in Iowa:
New
York-based political consultant Kieran Mahoney's statewide survey of probable
Republican participants in the 2008 Iowa presidential caucuses shows this
support for the "big three" GOP candidates: John McCain, 20.5
percent; Rudy Giuliani, 16.3 percent; Mitt Romney, 3.5 percent. Astonishingly,
they all trail James Gilmore, the former governor of Virginia, with 31 percent.
How could
that be? Because it was not a legitimate survey, but a "push poll."
That normally is a clandestine effort to rig a poll by telling respondents
negative things about various candidates. Mahoney makes no secret that his
voter sample was told of liberal deviations by McCain, Giuliani and Romney, and
of true-blue conservatism by Gilmore (Mahoney's client)
"Illegitimate" would be a clearly appropriate term had the sponsors
presented their results as a fair reading of current caucus preferences in Iowa. However, Novak tells us that Gilmore's
consultant "made no secret" that the survey question relayed information about
the "liberal deviations" of the other Republicans and "true blue conservatism"
of Gilmore. So while questions may remain about the poll's fairness, the
release itself is not quite as deceptive as Novak seems to imply.
Either way, the results as described do not add to a "push poll." That more nefarious
dirty trick is a truly "clandestine effort" to communicate to a mass electorate
with telephone calls made under the guise
of a public opinion poll (columnist Stu Rothernberg prefers
the term "advocacy call"). Rather, in this case, Gilmore's consultant
apparently conducted a legitimate poll that tested how a random sample of
likely Iowa
caucus goers would react to a set of messages that were (apparently) highly
favorable to Jim Gilmore.
Again, even if his terminology was sloppy, Novak was right to distinguish
this result from an attempt to measure current
vote preference. In this case, the Gilmore survey used what some call a "push
question" (others an "informed vote") to see how specific messages would move
the Iowa Republican electorate. Such exercises are common, legitimate tools used
by campaign pollsters to gauge the way voters will react to new information
received during the campaign. You can see many examples in the surveys
regularly released by the Democracy
Corps project of Democratic pollster Stan Greenberg.
Now, describing a poll as "message testing" does not let the pollster off the
hook. In some cases, pollsters work to make these questions as fair and
even-handed as possible. In others, their efforts may be fairly characterized
as "rigged" to produce results that work in one candidate's favor. And
sometimes - in a circumstance that brings joy to every campaign pollster - they
can do both at the same time.
So was the Gilmore question fair? It is hard to tell in this case,** because
neither Kieran Mahoney nor the Gilmore campaign are willing to share the full text
of the question with anyone other than Robert Novak. I spoke with Kieran Mohoney
today, and he explained that he let Novak see the results and the verbatim text
because he believed the columnist would see his characterizations of the
candidates as fair. Mohoney believes that Novak's review "constitutes as much
public validation as I'm interested in at this time," and politely declined to
release the text.
All of which brings me to three pieces of advice:
1) Be highly skeptical of results from an "informed vote" (or any other form
of "message testing") that does not include
the full, verbatim text of the questions. We have no way to judge the fairness
of the questions without reading their text. If you are a reporter, do not even
think about reporting such results unless you can see the full text, and
(ideally) point your readers to it as well.
2) A skeptical and critical read remains in order even when you do have access to the full text. Ask
yourself, are the candidate descriptions balanced? Do they provide only
positive information about one candidate and negative facts about the others? Do
the descriptions leave out important points that candidates will emphasize? Do
they give some issues an unlikely prominence? Remember, even a perfectly fair
"push question" attempts to predict the information flow of a real campaign,
and that is not an easy thing to do.
3) If you write about an unfair or biased result from a message testing
poll, please, please refrain from calling it a "push poll." The English
language leaves you many fine terms - negative, unfair, untrue, distorted,
biased, slanted and, yes, even "rigged" - that will describe an offensive poll with
far more accuracy.
**We can evaluate one aspect of this result, by reading between the lines a
bit. Novak's description implies a skew against other conservative alternatives to Giuliani, McCain and Romney,
such as Sam Brownback and Mike Huckabee, because their credentials were
apparently not described.
I have a Google News search for the term "push poll" which reliably produces items almost every day. More often than not, news items use the term --
inappropriately -- to describe a poll, or poll question, that someone finds
objectionable or biased. I have written about the subject often (most recently here, here, here
and here),
but I have to concede that Roll Call columnist and political commentator Stu
Rothernberg has created a simple,
concise review that ought to be required reading for any reporter, editor
or blogger before using the phrase "push poll" in a story.
To Rothenberg, it all boils down to the difference between
legitimate research and "advocacy calls:"
Polls are methodologically rigorous
public opinion surveys of generally 500 to 1,000 people intended to learn about
and measure voters' opinions and test possible campaign messages. Advocacy
telephone calls, on the other hand, are made to tens of thousands of people and
are intended to create or change opinion...
[snip]
As I have argued every year for the
past five and apparently will have to continue doing until I have taken my last
breath, push polls are really advocacy calls aimed at thousands of recipients.
They are like television or radio ads, except they are delivered over the
telephone. They seek to convey positive or negative information to influence a
voter's final vote decision.
Advocacy calls are not, in any shape or form, public opinion surveys.
Amen. But the hard part are those polls that seem to fall
somewhere in between. More often than not, Rothenberg notes, complaints about
"push polls" result from internal campaign surveys "that include very negative
information about a candidate for office." Here, he puts it plainly:
This kind of information can be
part of an advocacy telephone call or part of a legitimate poll. When they are
in a real survey, they are known as "push questions," because they seek to
measure which questions actually push voter sentiment and which issues can be
used by a candidate to win a race.
Push questions are not the same thing as push polls. Push questions, which are
included in a survey of only 500 to 1,000 respondents, are a legitimate part of
a public opinion poll that seeks to test effective messages.
You may not agree that "push questions" are legitimate or
ethical. Their content may be simply objectionable (depending on your politics)
or flatly untrue. True or not, such questions may anger respondents and the they
may produce deceptive results (if presented out of context). However, it is
important to distinguish between untrue or deceptive questions in the context
of a legitimate attempt to measure opinion and the sort of dirty trick fraud
that aims to broadly communicate a message under
the guise of legitimate research.
One last update (for tonight at least) on the "Push Poll" story we have been following today. TPMMuckraker's Justin Rood reports tonight on an interview with Zeke Smith of Common Sense Ohio, "the man responsible" for the calls into Maryland and Tennesee, and similar efforts in Montana and Ohio. In the interview, Smith confirmed that "his group uses a firm called ccAdvertising to make his calls," and offers a defense of their tactic. Surprise, surprise: They don't consider it "push polling."
He defended his group's questions ("Do you want your taxes raised?"). "Push polls" are used to spread negative information about a candidate, and are rarely used to collect respondent's answers.
The questions used "accurate characterizations," Smith said, and insisted his group was legitimately engaged in "data collection."
"There are a fair number of things that are unpleasant to talk about," Smith said. "But that doesn't make [our questions] any less accurate."
Listen to the recording of one of the Tennessee calls provided by Tom Woods of the Nashville Post and come to your own conclusions about the accuracy of their characterizations.
And then consider that we are dealing with a new variant of high-tech push polling. Like the push "polls" of old, there are no samples involved. They contact as many households as possible (how many calls do you think they had to make to reach two lines in Tom Wood's residence?). They use the guise of a public opinion poll to lure votres into listeing to the sort of distorted negative "messages" that benefiting campaigns will never publicly embrace. And then they add a new twist: As long as they cover it with the fig leaf of "data collection," all is legitmate.
Nonesense. This effort has nothing to do with research. It is about mass communication conducted under the false guise of a survey. Just listen to Gabriel Joseph of ccAdvertising talk about his services (as quoted by Daniel Schulman in Mother Jones):
"When you make 3 ½ million phone calls a day, we generally talk to more people than watch television, listen to the radio, or read the newspaper combined." He paused, then added quietly, "If someone writes something that I don't like, I can make their life—I can make them understand a few things if I choose."
We are getting more information about those push poll calls I posted on last night, the ones first brought to light by TalkingPointsMemo. While we do not know for certain who is responsible for the calls, a trail of circumstance points to one likely suspect.
Exhibit A: One of the aspects of the calls that struck me as both odd and unusual was that all the questions required yes or no answers, including the vote question. Again, from Pollster reader ST:
It was all yes/no questions. I assumed that yes or no was all the machine could process, because everything was asked in the form of a yes no question -- even who you were going to vote for. In the candidate preference part at the beginning and end you were asked would you vote for Corker (yes/no) and then would you vote for Ford (yes/no).
Real pollsters typically ask questions with more responses than just yes or no, especially vote preference. Why the odd format? It turns out that respondents did not answer questions by pressing the buttons on their touch-tone phones (the method used by IVR pollsters), rather they answered using speech recognition software that can hear a human voice saying "yes" or "no." I emailed ST and some other commenters who reported getting the calls. So far, one has responded to confirm that the call they received asked them to speak the words "yes" or "no."
Exhibit B: Listen to this sample "political survey" available on the web site maintained by ccAdvertising, a.k.a. FreeEats.com, a.k.a. ElectionResearch.com. All of the questions are asked in a yes/no format that utilizes, as the ccAdvertising About page tells you, their "patented (patents pending) Interactive Voice Response - Speech Recognition (IVRSR)." Listen to the entire "political survey," and immediately after the question asking if "you are undecided" in the race for New York Assembly, you hear the voice of candidate Charlie Fisher -- presumably a FreeEats client -- making a pitch for his election:
Hi, this is Charlie Fisher. I'll work hard to support your interests if you elect me as Assemblyman. I hope you'll vote for me on September 10.
Exhibit C: Back to the description of ccAdvertising's services it helpfully provides on its website.
ccAdvertising utilizes its patented (patents pending) Interactive Voice Response - Speech Recognition (IVRSR) method to ensure that our political, public policy and service organization clients have their messages reach the households they have targeted, usually based on location or anticipated household demographics [emphasis added].
Further down the page they point out that ccAdvertising "also engages in the distribution of market data research obtained in our public surveys." So these surveys serve a dual purpose. They "collect data" and they deliver "messages."
The problem, from my perspective at least, is that this message delivery capacity amounts to what the Market Research Association refers to as "Selling Under the Guise of Research" (or SUGGing): "a misuse of the survey process compromises legitimate marketing and opinion research surveys conducted by professionals," that "also causes distrust among the public and affects the reliability of all public opinion research."
Exhibit D: The article "Tales of a Push Pollster," out just last week by Daniel Schulman in Mother Jones. Schulman's article - definitely worth reading in full - is a profile of ccAdvertising/FreeEats. Here are some particularly relevant excerpts:
Today, FreeEats does mostly political work. In November 2002, the company issued a press release claiming to have played a role in the "Republican force that swept America on November 5," noting that "no fewer than six winning candidates and one hot ballot referendum were influenced" by its efforts....
Business has certainly been booming for FreeEats, which has deployed its technology on behalf of conservative candidates and causes ranging from the National Rifle Association and the anti-immigration Minutemen to Tom DeLay, who paid the firm $24,101 for telemarketing work between November 2005 and February 2006. DeLay's ally Grover Norquist, head of Americans for Tax Reform, has hired FreeEats to push his antitax agenda, including an unsuccessful effort to prevent a tax increase in Colorado.
FreeEats has also become the go-to firm for conservative groups fighting to restrict gay marriage and abortion, both issues that are dear to the company's chairman, Donald P. Hodel-a longtime Washington insider who served in two Cabinet posts (secretary of the interior and of energy) during the Reagan administration, then went on to become president of both the Christian Coalition and Focus on the Family. (Mother Jones' calls to Hodel's home in Silverthorne, Colorado, went unanswered.) In 2004, FreeEats was commissioned by the Defense of Marriage Coalition to promote a referendum banning gay marriage in Oregon. During the company's telephone surveys, Oregon residents reported being told: "In Massachusetts, where court-ordered same-sex marriage is legal, they are now preparing materials to teach the gay lifestyle to children, beginning in kindergarten." The referendum passed by a 14-point margin.
Now obviously, we do not know for certain who is behind the push poll calls reported in Maryland and Tennessee. But based on all the above, we can nominate a fairly obvious chief suspect.
UPDATE(1:40 p.m.): The Nashville Post has a story (via DailyKos diarist Rook) on "Common Sense Ohio," that group that has apparently sponsored the calls in Tennessee. Bigger news is that the story includes an audio tape of the call, so you can listen to it yourself. Try playing that mp3 side by side with the demo from ccAdvertising. The sound quality of the Tennessee call is poor, and my experience is that answering answering machine digital recording tends to distort the timber of voices. But it sounds to me like the announcer on the push poll could be the same voice as the announcer on the ccAdvertising demos. What do you think?
One big question about the recording: Was it caught by answering machine or voice mail or did the person who recorded it edit out their answers? If it was the former, then we have pretty conclusive proof of the intent of the push pollster, which is ultimately what defines push polling.If the questions continued in the absence of verbal answers, then the "pollsters" did not care one iota about collecting data. Their primary interest was communicating a message, even it if meant leaving unanswered "questions" on someone's voice mail. [Tom Wood, who recorded the call for the Nashville Post explains that he did in fact answer each question. See Update III below].
UPDATE II (2:18 p.m.): Pollster reader ST emails to say he believes the Nashville Post recording was edited to remove the respondents answers:
[The audio tape] is exactly what I heard. Some of the push language I didn't hear because I answered no to some topics. But when I answered yes, I got the Corker talking points. Notice that it says that the poll was PAID for by Common Sense Ohio.
Based on my experience, I think that someone edited out all their answers to that recording and it is NOT from an answering machine. Also, I can tell you they answered "yes" to each question. When I was called, I only answered "yes" to the question about taxes. To the abortion, gun and immigration questions, I answered "no." I got the pro-Corker push language "only" when I answered "yes." The call just skipped to the next issue when I answered "no."
UPDATE III (5:48 p.m.): After I contacted the Nashville Post seeking clarification of how the phone call was recorded, Tom Wood, the person who taped the call, posted the followng explanation in our comments section. He answers my question definitively. Yes, he says, he edited to audio to remove the sound of his answers to each question:
I recorded that audio for my colleague Ken's story, and yes, I did edit out my responses.
I had received the call on my downstairs line Saturday night, and I played along in order to hear what the questions would be. But on the question about whether I believe foreign terrorists ought to be allowed to live and work in the U.S., I responded with the first words that came to mind: "Fuck you!" The system then told me that the call would terminate unless I gave a yes or no answer. I finished the rest of the poll by giving answers as though I were a Ford supporter.
After that experience, when the call came in on my upstairs line yesterday, I grabbed my recorder and decided to answer as though I were a Corker voter. My responses prompted the system to give me the talking points on each subject, which my pro-Ford responses had not elicited.
Rather than try to explain that my responses were given for tactical reasons, I thought it would be simpler just to edit them. Sorry to have caused confusion.
Tom Wood Nashvillepost.com
Another reader, "Fisch," left a comment on last night's post that confirms that the calls required the respondent to provide some sort of answer:
I was literally dumbstruck by the question, and while I struggled for a response, the recording went on to the next question: whether I was in favor of striking the words "under" God" from the Pledge of Allegiance. Again, I was made mute by the sheer audacity of the questions. Then the recording announced the poll would end even if I did not answer all the questions. All I could say was "Good!" I was stunned by the call, I couldn't even remember the nature of the second question, until I read your post.
Finally, fans of the movie All the President's Men may appreciate the comment left below by Pollster reader Mark Patten. After receiving one of the calls he looked up the number for FreeEats.com and gave them a call:
I received one of the push-poll surveys in the Maryland Cardin-Steele race. I called the Herndon, VA number for FreeEats and asked the woman whom answered if this company was responsible for the robotic polling in the Maryland Cardin-Steele race. She answered yes, then became flustered, talked to someone nearby, then transfered me to another person. This was a young sounding man with a high voice and southern accent. Neither would reveal their name. I told him the number that I received the poll from (703-961-8297). He said that based on that number they were not responsible for this poll, but seemed evasive. He assured me that I would be on the no-call list, which was something I did not even bring up or know to ask for. Herndon, VA is indeed an area covered by the 703-961-xxxx number from which I received the poll.
Note: The number Patten says he "received from the call" is the number the push call left on his caller ID. Another commenter, Gail Powers, reports seeing the same number on her caller ID. The number, 703-961-8269, provides a constant busy signal. Google provides a listing for that number for an employee of a financial services firm in Virginia that appears to be long ago disconnected. I called the firm and they had no record of any employee with that name.
I wrote about allegations of push polling back in August. More often than not, the targests of these allegations turn out to be the internal message testing surveys conducted by campaigns rather than true "push polls" -- calls conducted under the guise of a survey intended only to spread negative information. Tonight, DK, the weekend blogger at Josh Marshall's TalkingPointsMemo.com, has been reporting (here and mostly here) on automated calls received this weekend in Maryland (and elsewhere) that certainly sound like the real thing.
Here is one example:
After asking you who you're going to vote for, it asks "do you want your own taxes raised or lowered?" Then it tells you that Cardin has voted to raise your taxes and will do so again. It follows with "do you believe the words 'under God' should be in the pledge of allegiance?" It tells you Cardin voted to remove them, which I assume is false. Then it goes straight to the gutter and asks "do you support medical research experiments on unborn babies?" Of course, it then tells you Cardin is for this. It finishes by asking again who you're going to vote for.
I am curious whether the recipients remember being asked any demographic questions, any attitudinal measures like ideology or party identification, any favorable ratings on the candidates, or questions geared at determining if the respondent intends to vote, is following news about the campaign or has voted in the past. If the "poll" asked none of these questions, but only the "questions" described in the quotation above, then given the timing, it almost certainly the sort of fraudlent "push poll" dirty trick worthy of the name.
I'll pass along further reports as find them.
UPDATE (10/30, 6:58 a.m.): Reader ST passes along this report from Tennessee
I have some experience in electoral politics and with legitimate polling, so I tried to pay attention as the call progressed
I was hit with the Tennessee version Sunday night around 6 p.m. EDT. First of all, they are interactive robo calls asking a series of yes/no questions.
You are first asked if you want to participate. Then you are asked if you would vote for Bob Corker. Same question for Harold Ford.
Next you are asked a series of yes/no issue questions. You get pushed if you answer a certain way. The first question was, roughly, do you want to keep your tax burden as low as possible. I answered yes to this one as was bombarded with a series of statements about how Corker advocates making the Bush tax cuts permanent, and how Ford wants to raise everyone’s taxes. Standard push technique.
They call then moved to other topics, asking if you would describe yourself as pro-life, asking if you support the NRA's strong defense of the Second Amendment, asking if you think there is a problem with illegal immigration in the U.S.
If you answered no, the call moved on to the next topic, if you answered yes, you got bombarded with pro-Corker talking points.
At the end the call again asked if you would vote for Corker, and if you would vote for Ford.
Then the call identified itself as coming from "Common Sense Tennessee" and gave a Web site commonsensetennessee.com. It also said it was associated with Common Sense Ohio and identified its treasurer as John Lind.
No demographic information was asked. It was all yes/no questions. I assumed that yes or no was all the machine could process, because everything was asked in the form of a yes no question -- even who you were going to vote for. In the candidate preference part at the beginning and end you were asked would you vote for Corker (yes/no) and then would you vote for Ford (yes/no).
Again, this call appears to fit the classic definition of "push polling," which is a fraud -- an effort to communicate a message under the guise of a poll -- not a poll at all. Real tracking surveys conducted a week before an election typicaly ask demographic items, attidues used to classify voters such as party identification, and usually track candidate favorable or job ratings. Real automated surveys are capable of handling questions with more categories than just "yes" and "no."
Thanks to our friend DemFromCT for emailing news of what sounds like a genuine "push poll."
Just last month, I wrote about allegations of "push polling" in congressional races in New York and Arizona. In that case, as in so many that appear almost daily in various newspapers across the country, the allegations sounded less like a true push poll (an effort spread information disguised as a poll) and more like internal campaign surveys testing how voters respond to negative information.
Today the Associated Press reported that a group named "Common Sense" has made automated calls to voters in Rhode Island asking about their preference in the Republican primary for Senate between incumbent Lincoln Chafee and challenger Stephen Laffey. According to the article, "those who chose Chafee heard graphic descriptions of an abortion procedure opponents call 'partial-birth abortion,' which the poll said Chafee supports."
The giveaway that this was a true push poll -- at least in my view -- came in the next few paragraphs:
Eva Geoppo, 57, of Providence, said she received four phone calls because she has multiple phone lines at home. On the first call she received, she chose Chafee when asked who she planned to vote for.
"It just freaked me out," said Geoppo, who owns a general contracting business. "They said something along the lines of 'Do you realize Sen. Chafee is for partial-birth abortions and he's a war monger?'"
The next time, she chose Laffey.
"It was 'Do you need a ride to the polls?'" she said.
Yes, a survey that samples randomly generated telephone number might, in theory, sample a household with multiple phone lines more than once. The odds of someone with four lines getting called four times, however, are astronomically small. Geoppo's experience implies an effort to systematically dial every phone in Rhode Island. Moreover, real surveys do not offer to give individual voters a ride to the polls.
According to the AP story, the Chafee campaign claims that the group behind the calls is "an Ohio-based organization" called Common Sense 2006, and officials of the group did not respond to requests for comment.
Interesting story. We will pass along more news if it develops.
Over the weekend, Greg Sargent of TPMCafe reported on what he considers "push polling, no question," involving some calls that trash two Democratic candidates for Congress, Kirsten Gillibrand in New York's 20th District and, more recently, Patty Weiss in Arizona's 8th District.
With all due respect to Sargent and his source, Mickey Carroll of the Quinnipiac University Polling Institute, both are using the wrong definition of "push polling." It is certainly more than poll questions that feed "the negative stuff," as Carroll puts it. A true push poll is not a poll at all. It is a telemarketing smear masquerading as a poll.
Back in February, in commenting on a different set of calls made, ironically, into the very same New York 20th District, I described real push polling in detail:
Many organizations
have posted definitions (AAPOR, NCPP, CMOR, CBS News, Campaigns and
Elections, Wikipedia), but the important thing to remember is that a
"push poll" is not a poll at all. It's a fraud, an attempt to
disseminate information under the guise of a legitimate survey. The
proof is in the intent of the person doing it.
To understand what I mean, imagine for a moment that you are an
ethically challenged political operative ready to play the hardest of
hardball. Perhaps you want to spread an untruth about an opponent or
"rumor" so salacious or farfetched that you dare not spread it yourself
(such as the classic lie about John McCain's supposed "illegitimate
black child"). Or perhaps your opponent has taken a "moderate" position
consistent with that of your boss, but likely to inflame the opponent's
base (such as Republican voting to raise taxes or a Democrat supporting
"Bush's wiretapping program").
You want to spread the rumor or exploit the issue without leaving
fingerprints. So you hire a telemarketer to make phone calls that
pretend to be a political poll. You "ask" only a question or two aimed
at spreading the rumor (example: "would you be more or less likely to
support John McCain if you knew he had fathered an illegitimate child
who was black?"). You want to make as many calls as quickly as
possible, so you do not bother with the time consuming tasks performed
by most real pollsters, such as asking a lot of questions or asking to
speak to a specific or random individual within the household.
Again, the proof is in the intent: If the sponsor intends to
communicate a message to as many voters as possible rather than measure
opinions or test messages among a sample of voters, it qualifies as a
"push poll."
We can usually identify a true push poll by a few characteristics
that serve as evidence of that intent. "Push pollsters" (and MP hates
that term) aim to reach as many voters as possible, so they typically
make tens or even hundreds of thousands of calls. Real surveys usually
attempt to interview only a few hundred or perhaps a few thousand
respondents (though not always). Push polls typically ask just a
question or two, while real surveys are almost always much longer and
typically conclude with demographic questions about the respondent
(such as age, race, education, income). The information presented in a
true push poll is usually false or highly distorted, but not always. A
call made for the purposes of disseminating information under the guise
of survey is still a fraud - and thus still a "push poll" - even if the
facts of the "questions" are technically true or defensible.
So it is not just about questions that "push" respondents one way or another, not just about being negative, not even about lying (although lying on a poll is certainly an ethical transgression). It is about something that is not really survey at all.
The calls that the Albany Times Unionreported do not fit the definition of push polling. First, the calls involved more than just a question or two. They included a series of "fairly innocuous questions," such as "whether the country is headed in the right direction," Bush's job rating and the initial congressional vote. Second -- and this is a big clue -- one respondent reports that he hung up in anger one night, "only to have a different person call back the next night asking him to finish answering the questions (he did)." That sort of "call back" is something a real pollster would do but a "push pollster" would never bother with. Third, the Times Union's reporting plausibly traces the calls to the Tarrance Group, a polling firm that has long conducted legitimate internal polling for Republican campaigns.
I am in no position to evaluate the substance of the attacks reportedly made in the calls in NY-20 or AZ-08, and I will certainly not try to defend them. The attacks tested in those surveys may well have been untrue, distorted or unfair. If so, they deserve the same sort of condemnation would we give if delivered in a television or radio ad or in an attack made in a debate. If the attacker is lying, it is unethical regardless of the mode. A television advertisement should not lie and neither should a pollster. But a lie alone does not a "push poll" make.
Is this just a semantic distinction? I don't think so. Just about every campaign pollster, Democrat and Republican, uses surveys to test negatives messages. If you think negative ads by Democrats, including theseexamples, were produced without benefit of survey based message testing, you're dreaming. If we choose to define a "push poll" as a survey that merely tests "the negative stuff," then we better be ready to accuse just about every competitive campaign of the same "dirty tricks."
If a pollster lies in a real survey, that's sleazy and wrong. If candidates distort the truth, let's call them on it. But if we confuse negative campaigning -- or the survey research that supports it -- with the dirty tricks of true "push polling" then we too are distorting the truth.
