Pollster.com

Articles and Analysis

 

Pew's Keeter: How Serious the Cell Phone Only Problem?

Topics: Cell Phones

During last month's conference of the American Association for Public Opinion Research (AAPOR), I mentioned the many papers being presented on the growth of cell-phone-only households and promised to report back. I have been digging through the many papers ever since, and have been working on some analysis on the subject which, in the crush of new charts and other activity around here, I have yet to post.

Fortunately, Scott Keeter, the director of survey research at the Pew Research Center, has posted a concise and accessible review of the cell phone challenge to surveys. The Pew Center has been at the forefront of research and development on this subject, conducting four pilot studies over the last two years that interviewed people with cell phones over their cell phones.

The summary is well worth reading in full, but for those in a rush, here is Keeter's view of where things are heading:

Pollsters recognize that some type of accommodation for the cell-only population will have to be made eventually, as was clear from the large amount of research on the topic presented at the AAPOR conference last month. In addition to the use of so-called "dual frame samples" such as those described above (calling both a cell phone sample and a landline sample), practitioners are discussing other alternatives, including the establishment of panels of cell-only respondents that can be surveyed periodically to track their opinions, and employing mail or internet surveys to reach the cell-only population.

For those who want more detail, I can also highly recommend the longer paper he presented at the AAPOR conference (co-authored with Courtney Kennedy and April Clark of the Pew Center and Trevor Thompson and Mike Mokrzycki of the Associated Press) which Keeter has now posted online. Of all the papers I have reviewed, it was easily the best review of the issues most relevant to the political surveys we all obsess over.

PS: When I return to the cell phone issue next week, I also want to say more about a question starting to bounce around the Internet (via Sullivan): "Are Cell Phones Killing Ron Paul's campaign."

 

Comments
Tom B:

Also, can you discuss whether anyone has thought about the fact that a 20 minute survey will take up valuable cell phone minutes that folks might not want to part with in order to take a long form survey?

____________________

Mark Blumenthal:

Tom B:

They have. That's the reason Keeter says about cell phone surveys: "It is also common practice to provide respondents with a small monetary incentive to offset the cost of the airtime used during the interview"

____________________

Gloria:

I look forward to your next post on Ron Paul. I believe that this is skewing Ron Paul's polling results down. He is hugely popular among young people and college students.
I'm not saying he has 20% support, but I don't think it's too off to say a few points higher in actual support, which would put him far and away above the other "lower-tier" candidates.

Another factor besides cell phones: VOIP landlines. I have a VOIP phone, and I'm not listed in the telephone directory because I don't subscribe from a traditional local serve provider. Even though I have a landline, I would never be called in a survey and neither would my boyfriend, who lives with me, or any of his libertarian friends or his grandfather, who all have VOIP service. More and more people have VOIP services, and I would bet many of them are the tech-savvy, young libertarians that are attracted to Ron Paul.

____________________

Gloria:

I look forward to your next post on Ron Paul. I believe that this is skewing Ron Paul's polling results down. He is hugely popular among young people and college students.
I'm not saying he has 20% support, but I don't think it's too off to say a few points higher in actual support, which would put him far and away above the other "lower-tier" candidates.

Another factor besides cell phones: VOIP landlines. I have a VOIP phone, and I'm not listed in the telephone directory because I don't subscribe from a traditional local serve provider. Even though I have a landline, I would never be called in a survey and neither would my boyfriend, who lives with me, or any of his libertarian friends or his grandfather, who all have VOIP service. More and more people have VOIP services, and I would bet many of them are the tech-savvy, young libertarians that are attracted to Ron Paul.

____________________

There are several problems with landline polling. You show me the poll, and I can show you the bias.


1) Sample Group Bias. At this point in the election, most polls that I have seen are calling random names from party registration lists. Thus Republicans who have stayed aboard the good ship "Dubya" sampling the kool-aid are the only ones being called.
In Iowa, if the person voted Democrat caucus last election, then there are not offered Republican choices, even if they are planning to switch parties, and vote in the GOP caucus.
And it's not just young libertarians who have only cell phones and VOIPs. Check out the comments here. Note the ages, the prior party affiliations, and the amount of money sent. These are not just young libertarians.
2) Sample size bias. Most of these MSM polls where Ron Paul is getting 1% are very small (300-500) In larger polls (~1000) he is polling 2-3% or better. Yes, the margin of error takes this into account, but most people don't pay attention to that, or even understand what it means.
If Candidate A is getting 12% and Candidate B is getting 25% and the margin of error is 6.5%, then who is ahead?
NO ONE - It's a DEAD HEAT.
But no one ever sees (or reports) that. These polls have high margins of error (due to small sample size) but no one in the MSM ever bothers to explain the math.
3) Reporting Bias. Most people or reporters don't realize that pollsters take several polls at the same time. Or campaigns contract with multiple pollsters. Which one we get to see depends in a large part on who is paying for the poll. Bad results are often not ever released.
4) Name Order Bias. If five candidates are running and the names are "rotated" in the following manner: ABCDE, BACED, CABDE, EABCD, DABCE - Then, clearly, candidate A is getting a huge benefit, and candidates D & E are getting the shaft even though the pollster can claim that the names were "rotated", and everyone got an equal shot at first place.
In the current case, the majority of polls reported to date DON'T EVEN INCLUDE Ron Paul. Not one "scientific" poll that has been done in Texas has featured Ron Paul.
NOT ONE!

Ron Paul is the only declared candidate being left out of this many polls (although they often include Bloomberg, F. Thompson and Newt Gingrich - who are not even running.) Brownback, Huckabee, and Tancredo are included in nearly 4 times as many polls as is Ron Paul. (19 vs. 5) Even T. Thompson is being included in nearly 3 times as many polls. (13 vs. 5)
5) Metric Bias. The polls don't really measure what they purport to measure (or, more accurately, what the MSM says they measure.) Modern telephone polls measure name recognition, and preference, not support or probable votes. Many Republicans would *prefer* that Julie-Annie be selected by the GOP it seems, but how many are sending him money, and how many will actually go vote for him once they find out his views on public funded abortions and political cross-dressing.
If you are given a poll which asks "which would you rather receive from an attractive member of the appropriate gender:
A) a slap in the face,
B) a kick in the crotch,
C) an oscultation on your oral fissure?"

How many will choose C? (a kiss on the mouth) Even if A wins in today's poll, how many will actually leave their couches on election day to go vote for everyone getting slapped in the face?
Neo-cons will say: "No one will vote Oscultation-of-the-oral-fissure (sure it's the best choice, but it's too esoteric, not enough people know what it means) You'd better vote Slap-in-the-face, otherwise you will split the vote, and Kick-in-the-crouch will win, and then we will really be in a world of hurt" - I really, really hate that stupid argument.
6) Time Bias. It is very early in the election cycle. We are over a year away from either nominating convention. A lot can and will change between then and now. In the example above, what will happen if three days before the vote someone spends a whole lot of money to buy ads to explain what "oscultation" really means?
Also once the poll is over, there is time bias in the other direction. The polls become out-dated and no longer relevant.
7) Commitment Bias. If A is getting 50% and B is getting 33%, but 60% of A's supoporters are not strongly bound, while 90% of B's are strongly bound, then B is more likely to win.
Focusing on the cell phones misses the larger problem with these traditional polls. I'm not saying that the internet polls are any better, or any worse. People self-select in landline polls as well. They decide to pick up the phone or not, and they decide to stay on the line or not.
Also, many landline polls correct for one or more of these problems where they can, but some (like lack of understanding about margins of error, and metric bias) are not problems with the poll per se, but problems with how they are presented to people by the MSM.
It seems to me that a properly designed internet poll, with demographic questions can be every bit as "scientific" as a land-line telephone interview, but can be fresher, and have more responses.

Later.

PS I don't mean to suggest that every polling company would stoop to biased name rotation, or multiple polling, to release only the one their client would like. Some of these problems are problems of the campaign, or involve only a few lesser companies who depend heavily on their political clients.

____________________

George:

The law says that you can't call anyone that would have to pay for the call, which is good. But with wireless providers frequently offering free in-network calls, it seems like RDD pollsters could reach a large chunk of cell-only voters just by identifying which network the phone number belongs to and calling in-network if it is a free call.

____________________

Paul:

I think this cell phone vs. land line issue is only part of the problem. As already mentioned, Ron Paul is the only candidate who has been in all 3 debates, whose name is deliberately left out of many polls, so no suprise that he doesn't do well in those. But I feel an even larger factor is that these polls are only calling registered Republicans, who were registered prior to the call lists being drawn up. Ron Paul has HUGE crossover appeal with Libertarians, Democrats, Independents, and quite a bit of that 70% of the entire US population who does not normally vote, but plan to this time for Ron Paul. Nobody is going to be calling all of those people, no matter what kind of phone they use, polling them about which Republican they are going to vote for.

Like Ron Paul has said, the base of the Republican party has shrunk. This is bad for the other Republicans, but not for him, since he has such huge crossover appeal. No other candidate of either party, has the crossover appeal that he does; especially no other Republicans. So as long as these crossover voters who are blown away by Ron Paul make good on voting for him in the primaries, the other Republicans don't have a chance. Except there are all these chumps who go "oh wow! That's Fred Thompson the Law and Order guy!"...

For fun, go to Youtube and search "debate Fox style". It is great to see how the FOX guys fawn all over Rudy G, slam Ron Paul, and then react in disbelief to the initial text message poll results...

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR