Pollster.com

Articles and Analysis

 

Dueling Obama-Palin Polls

Topics: 2012 , Barack Obama , PPP , Sarah Palin , Time/SRBI

Two new polls released yesterday asked about a hypothetical presidential contest between Sarah Palin and Barack Obama with very different results: The Time/SRBI poll (article, SRBI analysis & results) shows Obama with a massive, 21-point advantage (55% to 34% with 11% unsure or not voting), while a survey by Public Policy Polling (PPP) shows a dead heat (46% to 46% with 9% undecided). What gives? And which poll, if any, should we trust?

The two surveys were conducted by telephone just a few days apart (July 9-12 for PPP and July 12-13 for Time/SRBI) and sought to sample registered voters nationwide, but beyond those characteristics, they were very different surveys.

Time/SRBI uses live interviewers. PPP uses an automated, recorded-voice method that asks respondents to answer questions by pressing buttons on their touch-tone phones.

Their sampling methods are also very different: Time/SRBI used a method that selects telephone area codes and exchanges and randomizes the final digits of each number to theoretically reach a random sample of all working telephone numbers. In this case, they drew two samples, one of landline phones and one of cell phones, dialed each separately and combined the two with weighting. They attempted to select a random person in each household and ultimately questioned 1,003 adults (of whom 50 were interviewed by cellphone), although they asked the presidential vote question only of the 87% who said they are registered to vote.

PPP draws random samples of households from a list of all registered voters compiled by Aristotle International using the public lists gathered by voter registrars nationwide. Phone numbers are either provided by voters when they register or obtained by matching addresses to published phone directories, so some undisclosed percentage of the sampled voters lacks a phone number. PPP then interviews whomever answers the phone and asks respondents to “please hang up” if they are not registered to vote. They ultimately interviewed 667 registered voters.

The trade-off: Time/SRBI theoretically covers every registered voter, while PPP misses some undisclosed percentage of voters without phone numbers or that live in cell-phone-only households (federal regulations prohibit pollsters from using an “autodialier” to dial cell phone numbers). On the other hand, PPP’s identification of truly registered voters may be more accurate; self-reports tend to exaggerate the number of registered voters.

The PPP survey featured 17 questions, including demographics. The Time/SRBI survey asked 29 questions, not including demographics. Did the difference in length and mode (interviewer or no interviewer), produce a different response rate? Neither organization released a response rate, so we do not know.

Do all of these characteristics add up to different kinds of people interviewed? One big clue comes from the results for party identification: PPP’s respondents identified themselves as 39% Democrat, 34% Republican and 27% independent or other. Time/SRBI provided me with their party ID results for registered voters: On the initial question, 33% say they “usually think of” themselves as Democrats, 23% as Republicans, 30% as independents, 12% as “something else” and 2% were not sure. When they pushed the uncertain, a total of 47% identify or “lean” Democratic and 30% identify or lean Republican.

So the PPP sample has a closer partisan balance than the Time/SRBI sample, although we should keep in mind that the two surveys also asked slightly different party identification questions.

But wait, there’s more: The vote question also differs in an important way. Time/SRBI identifies the party of each candidate, while PPP omits party labels:

Time/SRBI: If the presidential election were held today and the candidates were Barack Obama, the Democrat, and Sarah Palin, the Republican, and you had to choose, for whom would you vote?

PPP: If the candidates for President next time were Sarah Palin and Barack Obama, who would you vote for?

Finally, the two polls asked their Obama-vs-Palin question in slightly different contexts. Time/SBRI asked their question following a set of probes of Obama’s performance as president, a question about whether Obama or George W. Bush “is the better president” and immediately following a job rating of first lady Michelle Obama. PPP asked their Obama-Palin question after a job rating of Obama, favorable ratings of each of the Republicans and immediately after a Huckabee-vs-Obama question.

The main point here is that these polls are very different in ways that go far beyond live interviewers versus automated polling. Their methods are dissimilar across the board.

So given these differences, which poll should we trust? My answer is neither. Or both.

First the case for both: When an attitude or preference is weak, small differences in methodology and question wording can make a big difference. Needless to say, asking for preferences in a hypothetical political match-up in 2012 two years before an election qualifies as weak. In such cases, it makes sense to look at a wide variety of polls in order to get a sense of the range of potential results.

The case for “neither” is a lot stronger in this instance, for the same reasons. Seven years ago while addressing the American Association for Public Opinion Research (AAPOR), my new boss Arianna Huffington offered a quip about a similar presidential vote preference question asked nearly four years before the 2004 election:

This is really about as meaningful as phrasing a question in the following way, which I will suggest you try one day, “If the world were to stop spinning, and all life were placed in a state of suspended animation, who would you like to see in the Oval Office when you thawed out?”

Yes, I’m guilty of sucking up a bit with that reference, but she’s right. How many ordinary voters have thought deeply about a contest between Obama and Palin? How many were forming an opinion on the spot when interviewed, only after hearing the question posed over the telephone?

My best advice to anyone trying to understand Sarah Palin’s potential is to put aside these two measures and focus instead on questions about opinions that are closer to real, such as Palin’s favorability rating (as asked in both the PPP survey and the results released by Gallup earlier today). Ordinary people do have genuine, pre-existing opinions about Barack Obama and Sarah Palin. Polls are on most solid ground when they measure these perceptions separately, rather than asking about a still hypothetical contest that has so far been of interest mostly to political junkies.

 

Comments
StatyPolly:

Thanks, Mark B. Excellent breakdown!

(the remainder of my most post is for at large readers)

How interesting. First of all, that Dem 47, GOP 30 party ID is not available in neither Time nor SRBI releases. It's only Dem+12 over PPP's party ID. Anyone thinks that may have anything to do with any part of the 21 point discrepancy? Hmmm? Perhaps about 12 of them? But what really exposes these narrative pushing partisan hacks is this:

In a wide open Republican field for the party nomination, Mike Huckabee (19%) and Mitt Romney (18%) are the early favorites, but preferences were scattered. Other Republican mentions include:

* Sarah Palin - 14%
* Newt Gingrich - 12%
* Jeb Bush - 9%

So Palin only gets an honorable mention. Fair, since she comes in pretty much a distance third.

BUT THEN WHY THE %$@$ IS SHE THE ONLY ONE POLLED HEAD TO HEAD WITH BOBO?

Ahhh, the venerable Time Magazine, the pillar of independence in journalism. LOW LIFES!

____________________

Ptolemy:

Although Gallup published them separately, the favorability poll of GOP presidential contenders (including Palin) and a poll of VP favorability ratings were both apparently part of the same survey.
http://www.gallup.com/poll/File/141272/Vice_Presidents_July_14_2010.pdf
http://www.gallup.com/poll/File/141335/GOP_Favorables_July_16_2010.pdf

Palin's favorable/unfavorable percentages are essentially equal to Al Gore. The question about Palin seems to have been asked just after the question about Joe Biden. Favorable ratings were about the same, but 7 percent had never heard of Biden, while only 2 percent didn't know Palin. The rest of the GOP field were into double digits for respondents that had never heard of them.

So according to Gallup, Palin comes in first for republican voters by a wide margin, first for a sample of all Americans, and at least even with current and former VPs.

____________________

Farleftandproud:

PPP was on drugs that day. Palin and Obama being tied, would shock me even on Rasmussen.

____________________

Farleftandproud:

I hope they nominate Jeb Bush for president, and we'll love to see his brother campaign for him. It would be a lot easier for Obama to give history lessons, and introduce a great war crimes cover up with CIA records, for Jeb to defend; it would force Jeb Bush to treat his own brother, the way Reagan had to shut out Nixon and those connected with him, when reagan had to repackage the Republican party. Jeb would be too much a thorn in the side of the party, just by "guilt by association".

____________________

Niikeb:

I think when looking more then 18 months out, the results are pure popcorn theater, Monday morning quarterbacking, or just for fun.

Right now there are 2 main poll #s that matter, Obama Approval and Generic Ballot. Once we get to September the campaigns will be in high gear and we can get a good view of any decisive wins that will happen. Harry Reid vs Sharon Angle really is semi-unpollable until Angle get's her name recognition near 100% and Reid bombs her. They both have tons of money, and right now very little has been spent.

So Sept-Oct till individual races start showing meaty data.

Also this is not a presidential cycle, so Generic Ballot should not move much for Sept-Oct, by that time it will all be individual districts that are up for grabs, not the national mood.

____________________

Farleftandproud:

It may not be a likely scenario, but this year there could be some surprises. I doubt Feingold will go down, yet he'll be close. Rossi could upset Murray and Paul could be upset by Conway.

____________________

chibeardan:

I didn't see the crosstabs for the Time poll, but PPP's poll asked how many people voted for Obama in the previous election. As I recall, around 46 or 47% of their respondents said they voted for Obama. It's certainly understandable that he has lost support among some of the people who voted for him, but looking at the hypothetical matchups, it only looks like he has lost 1-3% in any of the matchups.
So, since he won 53% of the vote in 2008, is it reasonable to conclude that with an erosion of 1-3%, he will likely garner much less than 50% of the vote? I'm confused by this weighting method, unless it is only being asked of voters who are likely to vote in the 2010 election. If that is the case, then I can understand the poll as trying to get a sense of this year's electorate. However, if it is actually trying to determine what the vote stands at right now for likely voters in 2012, then neither of these polls should be trusted - or at least a different weighting methodology should have been used by PPP, right?

____________________

Stuart Philipp:

Possible R losses in Senate:

FL

NH

KY

MO

. . .

____________________

quell:

Ummm....can you say voter enthusiam?
we have heard quite a lot about the conservative/liberal enthusiam gap.
I guarantee that putting Palin anywhere on the 2012 GOP ticket with bring Obama's base (youth and minorities and liberals) screaming down out of the heavens like a swarm of MQ9 Reaper drones on an afghan wedding party.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR