Pollster.com

Articles and Analysis

 

Update: How Accurate Were the Benchmarks?

Topics: David Yeager , Gary Langer , Harris Poll , Humphrey Taylor , Internet Polls , Jon Krosnick , Opt-in internet polls

Regular readers will recall that after ABC News polling director Gary Langer posted a new paper on a study conducted by a team of Stanford University researchers that found surveys conducted using op-in internet panels, we subsequently ran two guest posts by pollsters whose companies produce such surveys. One was from Douglas Rivers, the president and CEO of YouGov/Polimetrix (the company that is also the principal sponsor of Pollster.com).

A second came from Humphrey Taylor, chairman of the Harris Poll at Harris Interactive. Taylor's response argued that "social desirability bias" -- the notion respondents might not be comfortable accurately reporting on sensitive behaviors such as smoking and drinking -- might have caused errors in both the live interviewer telephone surveys and the in-person government surveys used as benchmarks in the paper authored by David Yeager, Jon Krosnick and their colleagues.

Today, Langer shares a response by Yeager and Krosnick to Taylor's critique that provides a long and detailed argument that the benchmark measures used in their analysis were not contaminated by social desirability bias. It is "perhaps not," as Langer notes, "the most casual reader's cup of Java," but is certainly worth your time if you read Taylor's guest post here in October.

As just about everyone involved seems to agree, these are serious issues for the survey research profession and worthy of this sort of high level argument. For those interested in a more basic review of the issues raised by the initial Yeager, et. a. paper, I devoted two columns to the subject in October.

 

Comments
poughies:

So at its worst internet polling has a 6% higher error rate? While on average is less than 2%?

If the reason for going to the internet is a. cheaper data. and b. data that could survive the cell phone age, then I'm not exactly worried. First off, if you believe the error rate is going to go through the roof with cell phones, then the 2% difference we see here will shrink. Second off, if the internet is cheaper, then you can take a lot more polls... and eventually you should be able to shrink that error rate overall through an averaging of the polls.

Not to mention that most internet polls aren't for political stuff, and I'm pretty sure people are just looking for general ideas of what are going on...

And keep in mind, I'm not the biggest fan of internet polling.

The charts on Langer's page are to say the least unconvincing to me.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR