Pollster.com

Articles and Analysis

 

Re: Obama Approval and the Battle of Interpretations

Topics: Approval Ratings , Barack Obama , Doug Schoen , Rasmussen

If you haven't yet read Charles Franklin's comments on yesterday's Wall Street Journal op-ed by pollsters Doug Schoen and Scott Rasmussen, click now and go read it. Then come back, because I have some thoughts to add, particularly on his observations on one sided interpretations and the power of partisan predispositions among polling analysts.

For those who can't wait, Charles' post addresses two big arguments from the Schoen/Rasmussen piece: First, that Obama's approval is now "below where George W. Bush was in an analogous period in 2001." Second, the headline (and sidebar graphic) arguing that "Obama's Poll Numbers are Falling to Earth" (which upgraded further to "Obama Cratering" when blogged by The Corner's Andy McCarthy). Although Obama's numbers approval numbers have declined, as Charles argues, both arguments rest on distortions or "one sided interpretations" of data.   

The biggest concern for Democrats should be the inevitable downward pressure from economic worries on Obama's job rating over the next two years. As Charles reminds us:

Any responsible economic analysis expects a significant lag before the stimulus has an effect, and assumes business conditions will improve after that for fundamental reasons as well. But not before the last quarter of 2009, at best. Ask Ronald Reagan how that worked out for him, as his approval sunk [from 59% in March 1981 just before the assassination attempt] to 43% around midterm election day 1982.

However, a few other aspects of the op-ed are worth a review.

"The American people are coming to express increasingly significant doubts about [Obama's] initiatives, and most likely support a different agenda and different policies from those that the Obama administration has advanced."

This line represents the central argument of the Schoen/Rasmussen op-ed. The two authors proceed through a litany of poll findings that illustrate public doubt about various Obama initiatives. Although many of these results should trouble Democrats, as Charles notes, the list is also relentlessly one-sided. Not surprisingly, Obama's pollster produced a similarly one-sided rebuttal of findings (blogged yesterday afternoon by George Stephanopoulos) showing support or enthusiasm for various Obama initiatives. The truth of public opinion about the specifics of Obama's agenda -- or perhaps more accurately about where public opinion is headed -- probably lies somewhere in between.

But one set of findings that Schoen and Rasmussen distort or leave out entirely involves questions that ask Americans directly about Obama's goals, policies and priorities:

NBC/Wall Street Journal (2/26-3/1, n=1,007 adults) - How confident are you that Barack Obama has the right set of goals and policies to be president of the United States: extremely confident, quite confident, only somewhat confident, or not at all confident?"
31% extremely, 23% quite, 26% only somewhat confident, 19% not confident at all

CNN/ORC (2/18-19, n=1,046 adults) - Do you think the policies being proposed by Barack Obama will move the country in the right direction or the wrong direction?
67% right direction, 31% wrong direction

CBS News/New York Times Poll. (2/18-22, n=1,112 adults) - Do you think Barack Obama has the same priorities for the country as you have, or doesn't he?
65% does, 28% does not, 7% unsure

I say "distort" because they do reference the NBC/Wall Street Journal question about halfway through their piece:

"[A]lthough a narrow majority remains confident in Mr. Obama's goals and overall direction, 45% say they do not have confidence, a number that has been growing since the inauguration less than two months ago."

The "narrow majority" is the 54% that say they are "extremely" or "quite" confident on the NBC/Wall Street Journal question. Fair enough. But 45% "do not have confidence?" Only if you interpret the 26% that are "only somewhat confident" to mean not confident. And the claim that the 45% number has been "growing since the inauguration?" Really? The NBC/WSJ poll asked this question only once before in early December and the results were virtually identical: 30% extremely, 24% quite, 29% only somewhat confident, 16% not confident at all. So 45% in December "grew" to 45% in late February.  

"Mr. Obama has lost virtually all of his Republican support and a good part of his Independent support, and the trend is decidedly negative."

I do not have access to Rasmussen's subscriber-only crosstabulations. I emailed Scott Rasmussen yesterday to request them, but have not yet heard back. If any readers are Rasmussen subscribers and want to post the most recent approval results by party, I will append them here [Update (3/15): Rasmussen's latest ratings by party are posted below].

However, we do have charts on Pollster.com for Obama job approval by party based on data from the seven national media pollsters that report results by party. These show that virtually all of the decline in Obama's approval rating has occurred among Republicans and independents, while Obama's numbers have improved slightly among Democrats.

Obama's current job ratings among Republicans on our trend estimate is 30.3% approve, 55.7% disapprove. Those numbers are certainly "polarized" in comparison to Obama's standing among Democrats, but approval from 3 in 10 Republicans is still long way from losing "virtually all" of his support.

Among independents, Obama's current rating is 55.5% approve, 27.4% disapprove on our trend estimate. Although Obama's disapproval scores among independents have more than doubled, his approval percentage has dropped by a more modest 6-7 percentage points since the swearing in. Yes, the trend is negative, but the decline to date is a far cry from a "good part" of Obama's support among independents.

[Update (3/15): Scott Rasmussen sends the following results by party, which come from the results released today based on 1,500 interviews conducted from Thursday through Sunday, March 12-14.  Not surprisingly, Obama's approval percentages among Republicans (23%) and in the "other" category (48%) are lower than those from other pollsters].

2009-03-15_rasmussen-by-party.png

Rasmussen: Obama Job Approval by Party


"While Congress's approval has increased, it still stands at only 18%."

That finding is from Rasmussen's congressional job approval question which differs from their presidential approval rating and produces lower scores than the question asked by other pollsters. The answer categories Rasmussen offers for the question about "the way Congress is doing its job" are excellent, good, fair and poor. This four point question format does typically produce lower approval scores when we combine excellent and good and compare to the percentage that "approve" on the classic two-category question.

Other pollsters that ask the traditional "approve-disapprove" question show higher recent approval scores for Congress (39% from Gallup, 41% from Fox News, 31% for NBC/Wall Street Journal, 26% from CBS/New York Times). Our current trend estimate based on all of these pollsters puts the Congressional rating at 35.4% approve, 55.1% disapprove.

Thanks to the reader that emailed with this observation and wondered whether the Rasmussen finding skews our congressional approval trend estimate. In this case, the impact is relatively small. Use the "filter tool" on our chart to remove the Rasmussen polls, and resulting estimates change by slightly less than a percentage point -- one point higher for approval (to 36.1%) and one point lower for disapproval (54.9%).

An aside: Rasmussen's presidential approval rating and its impact on our trend estimates are another story. Rather than the excellent/ good/fair/poor format, they ask respondents to choose from four categories of "approve" (strongly approve, somewhat approve, somewhat disapprove and strongly disapprove). Most other national pollsters offer just two categories (approve and disapprove). Also, on Obama's approval rating, Rasmussen has averaged just 2% in the "don't know" category, other pollsters have averaged 13%.   

Those two differences -- along with the fact that Rasmussen screens for "likely voters" and perhaps other, harder to detect differences stemming from their use of an automated methodology -- yield lower Obama approval scores than other pollsters. Obama's average approval percentage has been roughly 3 points lower on Rasmussen than on other polls, and his disapproval score has been roughly 14 points higher. You can easily see the difference in the disapproval scores in our chart below. There are two bands of red disapproval dots -- click on any of the higher numbers and you will see that virtually all are Rasmussen polls.

If we filter out the Rasmussen surveys on our chart, the impact on the current trend estimate is small for approval (roughly two points higher) but much more significant for disapproval (roughly seven points higher). The reason is both the size of the gap and the fact that the more frequent daily job approval tracking contributes 16 of 58 data points to our chart since the inauguration. Franklin and I have been discussing this issue recently and plan to blog more analysis on it soon.

* * *

Back to Franklin's observation about the "the power of predispositions" in polling analysis. I worked for more than 20 years as partisan pollster, but even as an "independent" blogger I still find it challenging to put blinders on and focus exclusively on the data. It is not easy, and I make no claim to perfection. Sometimes, despite the best of intentions, partisan predispositions seep through.

There is also a place for partisan advocacy. A one-sided, cherry-picked analysis by a Republican partisan appearing on the Wall Street Journal op-ed page is no more a surprise than a similarly one sided rebuttal issued by Barack Obama's pollster. What is surprising is to see that sort of advocacy coming from from two pollsters that explicitly identify themselves as "independent" and unaffiliated. Schoen and Rasmussen should know better.

 

Comments
GregAllman:

Thanks for taking the time to analyze this, Mark. When I saw this WSJ piece I couldnt believe the steps that were taken to build a (cherry picked) polling based case for the "imminent collapse of Obama's popularity" argument that the Hannity/Rush crowd have been espousing for weeks now. Rasmussen really went too far down the partisan alley on this one.

____________________

mk3872:

Excellent rebuttal here. While I found it very informative and the first definitive source to underscore Rasmussen as the ONLY natl pollster to create 4 categories of approve/disapprove to combine for only 1% respondents to be "unsure" ... I do however find it distressing for you to compare that to Obama's pollster's work.

Obviously Obama's pollsters are going to be one-sided! The difference is that Rasmussen is being positioned as independent and not affiliated with a political party or politician.

But this is untrue. Rasmusses has led some of the most blatantly right-leaning and consveratively intentioned polling in the past 12 months.

Just look up his latest polling and this breakdown by which shows what a political hack Rasmussen is.

____________________

gjdodger:

You know, the Right sure is infatuated by Obama's poll numbers and whether they reflect public dissatisfaction with his performance. When Bush's poll numbers tanked the last three years, they assured us the president was acting in what he felt was the public's interest and didn't care what the polls showed. Why is public approbation important to Barack Obama, but not George W. Bush? I suppose it's because the Right is deperate for evidence the ideologically rigid social and economic positions they've staked out reflect a significant share of the populace, and are not just a reactionary reponse to a geographically and culturally isolated minority.

____________________

The final Rasmussen 2008 U.S House election poll was
Democratic 47%
Republican 41%

The actual result was:
Total Vote: 120,662,065
Democratic: 65,203,062 54.0%
Republican: 52,290,917 43.3%

http://www.dailykos.com/story/2009/2/25/205426/346/845/701920

I don't know what that suggests of Rasmussen because I forget what the final numbers of the other polling firms were. Obviously they significantly undercounted the people who planned to vote Democratic for Congress (though they also undercounted those who planned to vote Republican).

Even adding in votes for the fact that the Republicans had many less candidates on the ballot barely changes the percentages. For those that want to read my calculations of that:

(However, this total covers 430 Districts. There were 5 incumbents (1 in Arkansas, 2 in Florida and 2 in Louisiana) who faced no challengers and did not appear on the ballot, they were reelected by acclamation. 4 of them are Democrats and 1 is a Republican.

If we assume that the winning candidate would have received 200,000 votes and a prospective losing candidate 50,000 votes.
We get an additional total of
Democrats 850,000
Republicans 400,000

This is offset though by the fact that the Democrats ran 22 more candidates than the Republicans in races on the ballot. Of the races on the ballot, Democrats did not run a candidate in 13 districts and Republicans in 35. If we impute each of those candidates 50,000 votes (this is purely made up numbers) we get an additional

650,000 votes for the Dems and
1,750,000 votes for the Republicans

which all adds up to 1,500,000 more votes for the Dems and 2,150,000 for the Reps. So, adding that to the initial totals we get. 'Other' would be down significantly because a lot of people who vote 3rd party do so in districts where one of the major parties doesn't have a candidate on the ballot.

Total 124,000,000 (approx)
Dems 66,700,000 54.0%
Reps 54,450,000 44.0%

Which barely changes the real official percentages at all.)

____________________

gjdodger:

Adam--here are RCP's final polls:

http://www.realclearpolitics.com/epolls/other/generic_congressional_vote-901.html

Three of the five were pretty close, all at 12% spread. The other two were as far off as Rasmussen.

____________________

Adam T:

Thanks for that. I thought I remembered other generic Congressional polls.

Odd that the RCP poll of polls doesn't include Rasmussen. I should look over other RCP polls. Is it because they officially exclude Rasmussen or did the Rasmussen poll just come out after all RCP publish that?

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR