Pollster.com

Articles and Analysis

 

"Common Sense" - It's Not a Survey

Topics: 2008 , AAPOR , John McCain , NY Times , Push "Polls" , The 2008 Race

If you were paying any attention to political news over the last few day, you have no doubt heard of Common Sense Issues, the group that has, according to the New York Times ,

begun making what it said were a million calls to households in South Carolina telling voters, according to one of the calls, that [John] McCain has "voted to use unborn babies in medical research."

"We hope to call 546,000 households in Nevada on behalf of Huckabee," said Patrick Davis, the executive director of Common Sense Issues, according to the Las Vegas Review-Journal.

Want to hear one of the calls? Here, via Ambinder, is audio (and video) of one of their calls as captured by a recipient:

Is this a so-called "push poll," an attempt to communicate an negative message under the false guise of a survey? Of course it is. But that's not the way Davis sees it, according to the Washington Post:

[Davis] questioned why McCain is characterizing the phone drive as an attempt to engage in push polling...Davis said the 45-second calls use a special technology that provides a different automated message, depending on how the recipient answers questions.

Moreover, Davis said, "A strict push poll is delivering not-truthful information. Everything we say is factual and backed up.

The Review-Journal adds:

Davis said the calls made by his group should not be called push polls because questions are asked of those called.

"A human voice is recorded asking the questions," he said. "You respond with your voice. How you respond dictates the next question. We are gathering information.

What a crock. The recipients that receive these calls are told they are participating in a survey, not a promotional message. Davis may have the high tech cover of asking questions, but the clear intent is to communicate negative messages. Check their web site's About Page. The expressed purpose of Common Sense Values is "educating and informing citizens in an in-depth manner about public policy issues." They say nothing about gathering data or measuring public opinion. They dress up their calls as "surveys" to add false credibility. Would the recipients stay on the phone if told they were about to be "educated and informed?" "Common sense" tells you they would not. The guise of a survey is a sham.

If you don't believe me, ask my colleague Nancy Mathiowetz, president of the American Association for Public Opinion Research (AAPOR):

Asking questions does not make it a survey. These calls are clearly a fraud that harm the survey research profession.

Interests disclosed: I am able to report a reaction from Mathiowetz because I am spending the day in a meeting fulfilling my duties as a member of AAPOR's Executive Council (so blogging will be light). AAPOR has a full statement condemning the so-called "push-polls" which groups like Common Sense Values conduct, and Mathiowetz devoted a HuffingtonPost blog to the subject a few weeks ago.

 

Comments
Leo Simonetta:

It is ridiculous to characterize the calls that "Common Sense" makes a survey or a poll. No survey or poll would ever call 500,000 households in one state much less a million. The purpose of a survey/poll is collect information while their purpose is to persuade.

____________________

At what point do phone calls like these either violate the do not call list law or even cross the line into slander?

____________________

Steve Hendricks:

Unfortunately, the calls do not violate the stipulations of the "do not call" regulations. Congress opted to exempt political telemarketing (along with charities) from the restrictions on such calls.

I'm not sure what solution exists for this problem other than trying (probably unsuccessfully) to distinguish between "polling" and "telemarketing" in the media and among the public.

____________________

C.S.Strowbridge:

Are you telling me someone who would use a push poll would lie about it? No! That simply can't be!

"I'm not sure what solution exists for this problem other than trying (probably unsuccessfully) to distinguish between "polling" and "telemarketing" in the media and among the public."

I think no survey would need to contact more than 10,000 people in any one period to get enough respondents. Perhaps the law could be adjusted with that in mind.

____________________

Nic:

We should not be calling these calls "push polls" at all. To use that term is to acknowledge that these calls have something in common with a "scientific poll" or even a "straw poll" We ought to be calling these calls "persuasion calls" or possibly "push calls." Still, as pollster, we need to push using another term for "push polls"

____________________

oh4real:

Hey - Wait a second.

While not scientific, aren't you dying to know what the response trees were?

I am quite certain that while they will never 'publish' their results, their IVR system database is no doubt logging the responses and they are no doubt able to formulate relative conclusions of citizen mood and trends.

It could be argued that their approach, IVR with a few direct/interactive questions with logged response trees, could result in polls with 10,000s of respondents that, when matched with census/voter roll data could give very nice overlays.

This would eliminate issue of losing majority % of respondents because they don't want to answer all of the demographic questions that provide cross-tabbing and drill down value. Keep the questions (non-push) down to just a few critical ones and sample size should grow dramatically.

Additionally, throw in last question of 'would you like to be contacted in person to explore your responses?' to build list of follow-ups which then you could proportionately (yet randomly) select from so that human calling efficiency would go through the roof (lower overall cost) and controlling how many respondents of each subset should improve representation and reduce error margins.

Using data analysis to correlate responses (& trees) to income/age/sex/voting/etc. demographics from other data sources would provide a generally useful view. And could be conducted more frequently.

When honest, only interested/motivated people will reply to longer human-polls anyway, this way less motivated might give response level data, then those that would answer human polls anyway would answer 'yes' to follow up.

Process:
5 critical questions in a response tree with yes no responses.
6th question 'zip code'
7th question 'follow up' yes/no.
Call time less than 30 seconds.
Cost pennies per call.

Back-office:
Crunch yes/no answers in tree and compare to external demographic data based connected via phone number/zip code (discount where they don't match).

Using general demo data and actual responses (including call me back) to generate pools of desired responses, then call back for drill-down and verification of relationship between demographics sources and ground-data.

First round is 'what', second round is 'why'.

Business model:

Freely publish the 'what', then charge for the 'why'.

I am an amateur pollster doing occasionaly market research surveys for my product lines, is this process already done normally?

Regards,
oh4real

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR