Pollster.com

Articles and Analysis

 

New Pew Data on Cell Phones

Topics: Cell Phones , Pew Research Center , Pollsters

The Pew Research Center, which has been at the forefront of efforts to measure the impact of "cell phone only" households on political surveys, has a new report out on subject today. Like Gallup they have found evidence that including interviews of cell-only Americans as a "modest affect" on results in the presidential race:

Pollsters are continuing to monitor changes in telephone use by the U.S. public, since most surveys are still conducted using only landline telephones. Growing numbers of Americans are reachable only by cell phone, and an even larger number who have both a landline and a cell phone may be "functionally cell-only" because of their phone use habits. The latest Pew Research Center national survey, conducted June 18-29 with a sample of 2,004 adults including 503 on a cell phone, finds that the overall estimate of voter presidential preference is modestly affected by whether or not the cell phone respondents are included. Obama holds a 48% to 40% lead in the sample that includes cell phones, and a 46% to 41% advantage in the landline sample. Estimates of congressional vote are the same in the landline and combined samples. [Emphasis added].

The numbers noted above are based on interviews with registered voters. When they narrow the universe to more likely voters, however, the difference [mostly] disappears:

Narrowing the analysis to voters who are certain about their vote choice, there is almost no difference between the landline and combined samples: Obama has a 38%-28% advantage in the combined sample, while the margin is 38%-30% in the landline sample.

For more detail on the challenge of cell-phone only households to political polling, see my two-part series last year, as well as any of our more recent posts on the subject.

[I added the word "mostly" to my second paragraph based on comments below. The report does not speak to the statistical significance of either set of numbers].

 

Comments
CaptainPatent:

This is an interesting result. If I would have ventured a guess on my own, the people I envision as a more "typical" cell-only household would be younger and in a more metropolitan area which - judging by many crosstabs I've seen should heavily favor Obama, but apparently that is not the case.

I guess there won't be an added and unexpected boost come November, but it is interesting to see results like this come in.

____________________

Am I interpreting the phrase "the sample that includes cell phones" correctly to mean the sample of 2,004 only 503 of whom were contacted by cell phone ? If so, backing out from the rounded results presented by Pew suggests that 54% of those contacted by cell phone support Obama and 37% support McCain from whole sample average equals landline + (cell phone - landline)503/2,004.

Now the variance of obama-mcain in the landline sample is less than 1/1,501 and the variance in the cell phone sample is less than 1/503(some people are undecided so for Obama and for McCain aren't perfectly negatively correlated). Bit more rounding up and the variance of the difference in the two differences is less than 4/1,503 so the standard error of the difference is less than
0.2/root(15) which is less than 5%.

The difference in the difference (using rounded numbers) is about 17% or 3.4 times the overestimated standard error. Now clearly one shouldn't round to two figures then do the calculations, but that is a very large ratio and makes me think that the results by cell phone are strongly statistically significantly more favorablet to Obama.

As far as I can tell from the snippet you quote, the 503 are not cell phone only customers or functional cell phone only customers. I should read the report (should have before posting this) but it seems that they are just people contacted by cell phone. Thus if one were to assume that the only difference between results with people contacted by cell phone and people contacted by land line is due to the fairly small estimated number of people who can't be contacted by a land line, such contactable only by cell phone people would have to be even more overwhelmingly for Obama than the 503 people contacted by cell phone in the sample.

I'd say that Pew is determined to conclude that they don't have to worry about cell phones which are a hassle and that the presented numbers which prove that they do have to worry in a misleading way.

____________________

Mark Lindeman:

"I should read the report (should have before posting this)"

Well, yes. A glance at the first table might be really helpful.

I'll hazard disagreement with your assessment of Pew's motivations.

____________________

brambster:

After a quick glance, this top line result seems skewed a little bit towards Obama.

In their cell phone only result, registered voters were just 58%, well below the 78% of land line users. The estimates of cell phone only households is 22.3% of all adults. So if you were looking for registered cell phone only users to add to the land line result, you would be using 13% in your sample as cell phone only registered voters.

What Pew did was combine the full sample and didn't adjust for registration. If you were to adjust for registered voters only, you would have gotten 919 land-line registered voters, and 139 cell phone only registered voters, for a sample of 1058 registered voters. This is the only way that makes sense due to the disparity in registration rates among these groups. I suspect that this would result in an even smaller difference between the two candidates since only 13% would be added, and the difference in candidate preference is uncertain in this RV group, but not likely to be any better than the 61 to 32 Obama advantage. This group is clearly very urban, young and minority, and the fact that 77% claimed to be absolutely certain to vote, that flies in the face of the 58% registration.

Considering the rapid growth in cell phone only households, it would be wise to start to include these people in samples, but I would hope that pollsters take care to adjust for registered voters when attempting to include this data if in fact Pew's results hold true on dramatically lower registration rates.

On a side note, it's a real shame that registration is such a barrier to voting. In my state the Republicans long blocked a motor voter bill until around the time that the US government required states to do this. Now we see Republicans pushing for poll ID bills, banning felons from ever voting again, and other things that limit those of lower socio economic status from exercising their rights to vote. We also saw in 2000 that Republican counties in Florida and elsewhere have equipment capable of alerting voters that might miss a vote or invalidate a vote, while Democratic leaning counties used equipment and ballot designs that invalidated over 20% of the presidential vote in one case (not Palm Beach, but a mostly AA populated county).

While our elections do represent mostly what those capable of and interested in voting prefer, our system does not represent the broader will of all people. If you were to encourage registration and voting instead of discouraging it, the movement to the more liberal leanings of the larger group would cause very fundamental changes to the political landscape since Republicans would be hardly competitive in many more states without adjusting their platform. Of course if we actually liked our government, people might vote more too.

____________________

Arnaud:

..."When they narrow the universe to more likely voters, however, the difference disappears:


The difference disappears? 38-28 and 38-30 is not equal.


____________________

sherman:

Not apropos of this particular item, but I am rather frustrated with the polls that show lower approval ratings for congress than the president. The polls never try to go into the details of why the approval is so low for congress, especially given the near uniform assessment that the public thinks that they would rather have more Democrats in congress than Republicans. Why don't we have questions that try to asses the reasons for such low congressional approvals? Is it because the Democrats didn't end the war? Is it because Republicans are obstructionist? Both parties spend money and talk BS? Come on! Tell us! Wouldn't it make a great news item if something came up that was unexpected?

____________________

Undecided:

FYI... The difference between 38-28 and 38-30 are "statistically" insignificant. No absolutes in statistics.

____________________

hobetoo:

I broadly concur with Waldmann's conclusion that the cellphone only respondents are far more disposed toward Obama than the other respondents.

But here's the rub, which a later commenter picked up indirectly: the cellphone only respondents differ not only in their having a lower percentage of registered voters but also probably in other demographic characteristics. In essence, the cellphone only respondents are likely to be young and also to include a disproportionate number who have never voted before.

Thus Waldmann's analysis ought to be supplemented by a multivariate regression that adjusts not only for registration but also for age, gender, & race (perhaps also education). The "test" for cell phone vs. landline would be more meaningful with controls for both demographic characteristics and registration.

Nonetheless, this doesn't negate Waldmann's general observation that those who can be reached only by cellphone have far different candidate preferences from other respondents.

Nor should the importance of this difference be explained away by saying well those cellphone only people only a small proportion of the voter eligible population or that they are far less likely to vote since 40% aren't even registered.

The reason for the latter comment has to do with the fact that the election is still 100+ days away, and the parties are conducting registration drives (though one of them is actively trying to scrub the voter lists by applying strict voter ID standards). The cellphone only population is probably the largest target group of yet-unregistered voters.

Therefore, for the campaigns and the polling organizations it is meaningful to calculate the difference in candidate preferences both for cellphone only and for other respondents -- and for unregistered as well as registered voters separately.

____________________

cinnamonape:

It would also be of interest to see if there were registration and voter preference differentials between those of similar demographics (age, urbanity) that used land-lines exclusively or partially. If one is going to massage the data on the cell-phone side re. registration status one should also do so on the other side.

As well, even those cell phone users that have a landline available may make far less use of it than their cell. They may be less accessible via landline and predominantly use the cell phone, or use their land-line for other purposes (fax, computer).

____________________

JimM:

I'm drawing a different conclusion - simply that Pew's sample size is insufficiently large to detect a significant difference.

The data suggests that, if there is an effect, it may be in the 2-3% range, within the MoE of this survey.

Further studies are required, including a large cellphone-only component. Need to knock the MoE on that subsample down to 3% or less.

(My stats books are at the office and not at home, not sure how large the sample needs to be.)

____________________

This issue will definitely be a wild card on election day. The fact that cell-only voters prefer Obama is not very surprising. VoteRobot.org

____________________

What about phones on the Federal "do not call" list? Are pollsters allowed to call those phones? It would seem to me that people who have placed their phone numbers on this list are likely to be more internet-savvy and thus more likely to be in Obama's demographic.

____________________

hobetoo:

On the theme of this discussion I'd also call attention to the recent article by Nate Silver on 538.com. He raises some questions about the perhaps hidden increased margin of error resulting from imputing the views of cellphone users from landline-based respondents.

Thus there is not just a potential "bias" (in terms of candidate preference) from the exclusion of cellphone only respondents but also simple error that can benefit either candidate and lead to inconsistent results from one poll to the next by the same pollster in the same state.

Perhaps some sort of "poll volatility" indicator, comparing, say 2004, 2006, and 2008, would reflect this, for pollsters who claim to adjust for cellphone only non-respondents.

Here's a link to the 538 article:

http://www.fivethirtyeight.com/2008/07/cellphone-problem-revisited.html

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR