Pollster.com

Articles and Analysis

 

Will Party Weighting Account for the Cell Phone Only Problem?


The key takeaway from the recent Pew Report on cell phone only users was not that cell phone only respondents are different, but that even weighting landline only survey data doesn't fully account for excluding cell phone only users. Typically, a survey may be weighted for factors such as age, race, gender, education, and region. This allows pollsters to take a particular sample, and adjust it to look more like what they think the population they are interested in actually looks like. Pew found that even if you weighted a landline sample for all of these factors, that sample still provided results that were 2%-3% less favorable for Obama than one that included cell phone only users.

What exactly makes cell phone only respondents different from those with a landline? If it is simply the case that cell phone only respondents are more likely to be Democrats than those with landlines, then it should be simple enough to correct for not calling cell phones by weighting a sample by party identification. From my reading of the Pew report, they did not examine whether applying a party weight would have accounted for the exclusion of cell phone only respondents. A party weight is something that some pollsters (like Rasmussen) apply, but others do not. However, based on some recent analysis I have conducted using the 2006 Cooperative Congressional Election Study, even weighting by party is not likely to fully account for the differences between cell phone only respondents and those with landlines. There are two reasons for this.

First, it is true that cell phone only respondents are more Democratic than landline respondents. But this relationship is a little more complex than it first seems. In the 2006 survey, cell phone only respondents were just 4% more Democratic than landline respondents and they were 9% less Republican when asked a standard party identification question. However, once you factor in independents who lean towards the Democratic or Republican Party, you find that cell phone only respondents are 10% more Democratic and 12% less Republican. Thus, the party differences are larger when you factor in leaners, a pattern that results because cell phone only respondents are more likely to initially call themselves independents even though they lean Democratic. To fully capture party differences among cell phone only respondents, one would need to factor in leaners.

pidcellonly.PNG

Second, just looking at party affiliation masks the fact that cell phone only respondents are actually quite a bit more liberal than those with a landline. 35% of cell phone only respondents classified themselves as liberals compared to just 23% of those with a landline. These ideological differences are not completely accounted for by party either. From the table below, you can see that cell phone only Democrats are 10% more liberal than those with landlines. Democratic leaners in the cell phone only sample are 15% more likely to classify themselves as liberal. And even those cell phone only independents who did not express a lean to either party were more likely to be liberal compared to their landline counterparts. Given that cell phone only Democrats and Democratic leaners are more liberal than those with landlines, they should be less likely to defect and vote Republican than landline Democrats.

libamongdems.PNG

Thus, this analysis suggests that differences between cell phone only users and those with landlines cannot simply be accounted for by partisanship. In fact, even when I used multivariate models controlling for a wide range of demographic and political factors (party, age, race, gender, income, education, and even religion), cell phone only respondents were still substantially more liberal than those with landlines. Cell phone only respondents are ideologically distinct in ways that cannot be accounted for by party identification or all the other standard demographic factors that pollsters may use to weight samples.

 

Comments
Adam:

There's no mention here of age. Could the differences in party ID and "liberal" ID really just be reflecting the leanings of younger voters?

____________________

Disco Stu:

@Adam:

He does mention age in his analysis.

I think another thing to consider is CPO's (Cell Phone Onlys), even if they are Republicans, may be more urban...which likely means more fiscal conservative than socially conservative....and in this environment, McCain's "base" is the social conservatives.

Of course I have heard than many of the polls in the primaries were spot on...maybe the CPO effect is countering what's left of a Bradley effect?

____________________

THX1138:

From empirical, personal experience, age is a huge factor. Living in Boston (granted, a deep deep cobalt blue state!) - I would say 90 percent or more of my friends under age 30 have ONLY a cell phone.

Question - will pollsters in future elections be able to tap into cell phone lists as well as land lines, avoiding this CPO issue??

____________________

I'd like a similar analysis done for internet only. I have a suspicion that older people who use internet are wealthier and thus more Republican than their non-internet peers. In this case, weighting would not solve bias among internet polls. Pew occassionally runs internet usage questions. Perhaps they - or some other organization- could look at this in greater depth.

____________________

Adam:

Age is only mentioned as one of a slew of other factors that were fed into "multivariate models"; I still find it hard to believe it's not the main driver of any differences among CPOs.

____________________

DanManSF:

What about the Caller ID factor? I have Caller ID and I never answer my phone if I see "research" or "survey" in the ID --

____________________

Isher:

@DanManSF & Adam

Most of that is covered in the Pew research linked at the beginning - they are very thorough.

____________________

UpstateProgressive:

I keep wondering if this is the year when the traditional methods of polling hit the wall. Traditional likely voter models held up reasonably well in 2004 and 2006 but may seriously break down this year--it's hard to know if this will be the year or not.

It seems to me possible that, while Obama is leading in the polls (and Dems are expecting big win in the Senate & House), the polls are actually seriously underestimating the size of the Democratic wave. This would turn out to be true if the polls' likely voter models are underestimating youth turnout and AA and Hispanic turnout and not catching the cell-phone user effect. There is also the possible effect of the Obama ground game and the overall high enthusiasm of Dems and the low, low enthusiasm of the GOP.

If all of these factors hit at once, it could maybe make a difference to just 1% in the polls (still enough to turn some close states or races)--but hard to tell from statistical noise. But it also might make a difference of 2-4%, which would be a whole different story. That could turn lock in states like Ohio, FL, NV, VA and NC, make IN and MO hard for McCain to hold, and bring states like GA and MT into range. This could create something like a monster Electoral vote landslide and throw wide open the chance for 60 Dem seats in the Senate.

I don't think anyone now knows if this will happen. But it could, and if so, not only will Obama be president with a powerful mandate and a large working majority, but pollsters will need to find a new way to assess the voting patterns of the American public.

____________________

Espion:

My husband and I have only cell phones. No land line for 5 years now. We live in Virginia. I'm 31; he's 36. We're both voting for Obama. We generally vote Democratic. No one has polled us this year or any other year. Our voices don't show up in any polls but they will on election day.

____________________

muckinello:

@Espion:
Good work, turning Virgina blue this year! I will do my best here in PA to make sure no vote gets lost on Nov. 4. Let's win this thing for our country!

____________________

lisa r:

Like Espion, my husband and I have been cell-phone only since 2002. We're 'middle-aged' (40s), in Texas, and voting for Obama.

The age bit may not be as pertinant as a certain mindset or lifestyle. We're both techies.

____________________

nlbknitter:

One thing about cell only users is that more and more of them primarily use texting to communicate. Also, if they don't feel like answering the phone, they just ignore. How are you going to poll people like that! It does, however, characterize them as very indepedent minded.

____________________

drinkwine247:

Hey , here is a great article about how full of crap repubs are about Barack never working across the aisle and going against his party.....http://obsidianwings.blogs.com/obsidian_wings/2008/10/taking-on-his-p.html

____________________

As a 41-year old cell-phone-only user, it's delightful to see CPO users described as "overwhelmingly young."

And as far as the 6% Bradely effect: remember that Obama is half-white so his Bradley effect is only 3%

____________________

I'm a 49 year old white woman. Cell phone only. Nobody has called, although I'm in Massachusetts so maybe they figure it's not worth it. But it's still one more for Obama. Someone who doesn't understand or even use the internet is not fit to preside over a country that will have to decide issues of net neutrality and cyber-terrorism.

____________________

pbcrunch:

Rich Merritt:

No, you apparently don't go to freerepublic.com enough to know the true Bradley Effect. In formula form:

X + 3%, X = Obama's current lead.

Glad to clear that up for you!

____________________

bent1985:

I'm still not convinced that the age demographic of people voting this year will actually change. I may be wrong, but I will be very impressed if age breakouts of voters actually move substantial amounts. Ds have registered tons of new young and minority voters this year but (Republican vote caging aside) will they actually turn out???

Whatever happens this is going to be one exciting election for polling/statistics/trending junkies!!

____________________

cinnamonape:

There are a lot of reasons young people have cell phones. Since they are more mobile in terms of residency they can easily take their number with them. Many live in roommate situations, and they don't have to deal with negotiating bills. Many are going to school and work...so are away from their residence long hours. Again if they have a roommate they don't have to worry about the lack of privacy sharing an answering machine entails. If they move back home from college during the summer they don't need to pay a big deposit repeatedly to some local carrier.

This is why young people, particularly college students or those that have moved out of their parents households have gone to 100% cell-phone usage.

Even compared to their peers who have home phone access these individuals would likely be far more liberal in their attitudes.

____________________

cinnamonape:

I did a thesis on the original "Bradley Effect".What is commonly neglected is that, contrary to the myth, poll respondants did NOT lie to the pollster and falsely state support for Bradley when they were intent of voting against him.

Bradley actually got 2% more than what the polls suggested he had. The problem was that he picked up only about 25% of the "Undecided" voters. 75% of those swung to Deukmejian. And that "Undecided" subsample was fairly large (about 8-10% a week before the election).

Exit pollers also noted large numbers of Decline To State out of the polls...this was particularly true in Orange County and the San Joaquin and Sacramento Valleys. So they very likely received a disproportionate number of Bradley respondants from LA and the Bay Area.

It's not people who actually "report" to pollsters as "decided" that is at issue...it's a disparate/disproportionate "Undecided" or Decline To State vote that makes up the Bradley Effect. Thus if Obama draws above or close to 50% in any State poll the risk is less for this to be involved.

In addition, Reagan heavily campaigned for Deukmejian in California, particularly in Orange County and the Central Valley in the 1982 race during the last weeks of the campaign. THAT likely caused some voters to decide for "The Duke".

This Undecided "Bradley Effect" may play a role in some strong "Red States" (TX, AB, AR, MS). But there are many differences with the current situation. Red Staters may vote for a third party candidate if they become disenchanted with McCain. There's unlikely to be any waft of energy like Reagan sweeping into the state to buoy the hopes of Republicans. Deukmejian was the "outsider" after several years of Jerry Brown. There was a resplendant Republican party movement...rather than one in decline.

____________________

THX1138:

Thank you cinnamonape(?) ! Great analysis.

What does this bode for the number of undecideds this year??

If there is a 75/25 break towards McCain from those undecideds (because of Palin/McCain propaganda and hate-mongering) in the final days, what will that do to the narrow-margin swing states??

____________________

cinnamonape:

THX1138-

That's precisely why I mention the 50% threshold as being critical. As Obama approaches that he gains "immunity". The Bradley Effect is real...but it plays to the Undecideds. If Obama has a set number still supporting him (and few people shift in the last few days) above 50% then he would be safe.

It's really difficult to measure the impact of the "attack ads" on those "Undecided Voters". If you ask them they'll say that they are "turned off"...but whether that a) reduces the likelihood of voting, b) turns them away from the one pushing the ads, or c) subconciously leads them to vote for the "attacker" is a matter of dispute.

One argument is that rather than doing (c) it drives down the turnout of Undecideds who would vote for the "clean candidate" while increasing the turnout of "the base".

____________________

DrSpin:

One thing I think a lot of people miss in the discussion of CPOs is the number of lower-income people that are surely included in that group. With Tracfones available at grocery stores as cheap as $10, going CPO is not just about mobility or a certain lifestyle, it's just simple economics. I was CPO for several years because I just couldn't afford a landline (only have one now thanks to Vonage). The distribution of CPOs across the economic spectrum is probably bi-modal, with large numbers in the below-middle-class range and the upper-middle-class range. That's probably why controlling for variables like age, income, race, etc., doesn't account for ideological differences, because CPOs are not all young, rich, white, etc.

____________________

DrSpin:

cinnamonape, when and where did you do your thesis on the Bradley Effect? I'm writing a paper about media discussion of the Bradley/Wilder Effect for an academic conference and I'd be interested in reading more about your analysis. You can contact me at dunnsw@email.unc.edu

____________________

snipsnop:

Did the Pew people take into account folks (like me) who *have* a landline but never answer it? I only use the landline for long-distance calls when I'm running low on cell minutes, and for emergency calls -- enhanced 911 can locate the source of the call even if the caller (me) doesn't speak.

So I'm not a "cell-phone-only" voter, but I'll never be included in a landline-only survey.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR