Mark Blumenthal | November 18, 2008
Nate Silver has taken Zogby International to task for a telephone survey of 512 Obama voters that claims to "gauge their knowledge of statements and scandals associated with the presidential tickets during the campaign." Silver is right....but not right...
The summary and statement posted on the Zogby web site claims that "only 54% of Obama voters were able to answer at least half or more of the questions correctly," and more specifically that "statements linked to Republican presidential candidate John McCain and his vice-presidential running-mate Sarah Palin were far more likely to be answered correctly by Obama voters than questions about statements associated with Obama and Vice-President-Elect Joe Biden."
The survey was paid for by John Ziegler, a former talk radio host and publisher of a conservative web site. The Zogby summary quotes Ziegler claiming that "the poll really proves beyond any doubt the stunning level of malpractice on the part of the media in not educating the Obama portion of the voting populace."
The problem, as Silver points out, is that the survey does no such thing. It proves only that Obama voters surveyed were less likely to attribute to Obama or Biden a half dozen statements that were "at best debatable, yet apparently represented as factual to the respondent," such as the following:
"Which of the four [candidates] said his policies would likely bankrupt the coal industry and make energy rates skyrocket?"
"Which of the four [candidates] started his political career at the home of two former members of the Weather Underground?"
"Which of the four [candidates] quit a previous campaign because of plagiarism?"
"Which of the four [candidates] won his first election by getting opponents kicked off the ballot?"
Silver concludes -- appropriately -- that Zogby's survey appears to be less an unbiased measurement than part of "a viral marketing effort to discredit the intelligence of Obama supporters."
Zogby's defense is to deny that he conducted a "push poll" (more on that below), claiming instead that his survey represents "a legitimate effort to test the knowledge of voters who cast ballots for Barack Obama" He claims that "respondents were given a full range of responses and were not pressured or influenced to respond in one way or another." That's a little like describing the question, "when did you stop beating you wife," as fair (and as a fair test of "knowledge") by saying the husband has an opportunity to offer any date on the calendar as a response.
Zogby also claims to be a passive agent that just conducted research on behalf of a client. "The client," Zogby writes,"is free to draw his own conclusions about the research, as are bloggers and other members of society." Really? Then why does the analysis posted on Zogby's website repeatedly support "the client's" conclusions?
Unfortunately, Silver's case would have been stronger had he not reached reflexively, as so many do, for the "push poll" label to describe the Zogby poll. That's a bit like confusing assault with murder. A push poll isn't a survey at all, but negative telemarketing calls made under the guise of the survey. And the Random House Dictionary definition that Silver linked to is at odds with those of the American Association for Public Opinion Research, the Council for Marketing and Opinion Research, and the American Association of Political Consultants, (see also the work of Stu Rothenberg of Roll Call, Kathy Frankovic of CBS News and yours truly).
Zogby's survey does not amount to a "push poll" in that sense, but using the term allows him to respond -- predictably -- with a denial that "this was not a push poll." It wasn't, but that's beside the point. Describing his biased, leading questions as a legitimate test of knowledge is hugely misleading, at best.