Pollster.com

Articles and Analysis

 

Bialik on IVR Polling


"Numbers Guy" Carl Bialik devotes his Wall Street Journal column and a companion blog post today to the subject of the automated "interactive voice response" polling that has become such a staple of the current campaign. Both are well worth reading in full.

Bialik managed to interview most of the major players in the political IVR field, and had a reaction from our partner Charles Franklin, summing up our own philosophy regarding the automated polls (that use a recorded voice rather than a live interviewer, and ask respondents to answer questions by pressing keys on their touch-tone phones):

The automated-polling method, says Charles Franklin, professor of political science at the University of Wisconsin and co-developer of the poll-tracking site Pollster.com, "can prove itself through performance or it can fail through poor performance, but we shouldn't rule it out a priori."

The column notes that IVR pollster SurveyUSA ranks second most accurate among all pollsters during the 2008 primaries in the ratings compiled by Nate Silver and that IVR polling was indistinguishable during the primaries in terms of how the final poll compared to the election result:

Their accuracy record in the primaries -- such as it was -- was roughly equivalent to the live-interviewer surveys. Each missed the final margin by an average of about seven points in these races, according to Nate Silver, the Obama supporter who runs the election-math site fivethirtyeight.com.

Franklin did our own compilation of polls conducted during the final week of the 2006 (for a paper presented at the AAPOR conference last year) and reached essentially the same conclusion.

The article also indicates some cracks may be forming in the intense skepticism that the survey research establishment has long held for IVR surveys. Bialik notes that a polling textbook (The Voters Guide to Election Polls ) authored by Paul J. Lavrakas and Michael Traugott, "refers to these surveys as Computerized Response Automated Polls -- insulting acronym intended." But at the end of the column, Lavarakas indicates a willingness to consider the methodology:

Accepting responses by touch tones may have a particular advantage this election, says Mr. Lavrakas, former chief methodologist at Nielsen Media Research, because it may extract more-honest responses from white respondents about their intent to vote for Sen. Obama. "Ultimately the proof is in the pudding, and those firms that use IVR for pre-election polling and do so with an accurate track record should not be dismissed," he says.

Again, this is good stuff. Word reading in full.

 

Comments

I would love to see a comparison of self-identified partisanship between phone and IVR polls. The assumption that I am working from (and evident in some other data I am analyzing) is that the social desirability bias induces people to answer more frequently that they are an independent when being interviewed by a human.

This makes me wonder if the much discussed decline in self-identified Republicans in the electorate is a consequence of social desirability bias. As Republican Representative Tom Davis famously said, if the Republican Party was a dog food, it would have been removed from the shelves. If people are unwilling to say to an interviewer that they consider themselves a Republican when they really are, they might be more willing to identify as a Republican in an IVR poll. The problem - and what I am seeing in these other data - is that these are still mixed interviewing modes, so we are not fully controlling for everything that is different about these polling techniques when making this comparison.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR