Pollster.com

Articles and Analysis

 

Job Approval, Outliers and Perspective


ObamaAppoval.png
An absurd amount of attention has been given today to a new Zogby internet poll. In a very well managed PR campaign, Zogby leaked the results last night before releasing them a bit after noon today, while also debuting a Zogby presence on Twitter. A little Googling will turn up many hits citing the not-yet-released-at-the-time results. Nate Silver at fivethirtyeight.com helped the poll get attention by attacking it before it was even released. The right is delighted-- "Obama approval drops below 50%!!" The left (with Silver's help) denounces the poll. Twitter is abuzz on both sides. And media, both old and new, couldn't contain their chatter over the results. (My favorite, the Boston Herald's Joe Dwinell's lede "The honeymoon is over, a national poll will signal today as President Obama's job approval stumbles to about 50 percent over the lack of improvement with the crippled economy." The fourth paragraph notes that "Some polls show Obama coasting with a 65 percent job approval, but not in Zogby's tally.") 

It takes five seconds to put the new poll in perspective. Take that long to look at the chart above. The Zogby poll stands out pretty clearly in the chart, no? 

To make a lot of the Zogby poll is to deliberately ignore the context in which it appears. Yes, the left is correct that it is an outlier. A huge one. The right looks desperate or ignorant by embracing this result as meaningful. 

But rather than suppress the poll or ban it from polling analysis, let's just put it in the chart and let the data speak! No stats needed to get the point. 

Poll results are far more meaningful when we look at them with a bit of perspective. The chart also makes clear how the Gallup and Rasmussen daily tracking polls differ from one another as well. Gallup's daily tracker consistently results in higher approval ratings than do non-daily national polls. Rasmussen consistently results in lower approval ratings than do non-daily national polls. But put them all together and we get a trend estimate based on all the polls that matches the trend for conventional non-daily polls quite well. If we pick only Gallup or only Rasmussen, we bias our understanding of the state of public opinion. Not by a huge amount, but by a persistent two-or-three points up or down compared to the overall trend or to the non-daily polls. This is no surprise. House effects, the tendency of polling organizations to produce modest differences from one another, are well known and much written about. By putting all the data in the chart for everyone to see we let these house effects stand out in an obvious way. 

If someone still wants to cherry pick a result to suit their partisan preferences, then fine. That's called politics, duh! I'm not against it. But let's not confuse willful distortion for partisan purposes with analysis of what the data actually show. 

The Zogby internet polls have a considerable history that I won't repeat here. The point is not to condemn them but to put them in perspective. That done, I don't think much more comment in required.

 

Comments
RS:

So Gallup's a little high, Ras is a little low... Zogby is just erratic! :-)
So... will we see a Zogby op-ed in the WSJ about how Obama's crashing-and-burning?!

____________________

hobetoo:

No, Zogby isn't "just erratic." It's consistently way low.

____________________

OGLiberal:

A few things here...

First, as noted by the chart above, Zogby has consistently been an outlier.

Second, Zogby has 49% saying Obama is doing an "excellent or good" job and 50% saying he is doing a "fair or poor" job. Where I'm from, fair isn't bad. It isn't necessarily good and it's certainly not excellent, but it's not bad. How many of that 50% responded with "fair"?

Third, Obama's "favorability" rating in Zogby's poll only dropped 1pt, from 56 to 55. (Zogby's favorable ratings for the president are also an outlier, btw.)

Fourth - The right direction/wrong direction numbers continue to improve. In early January, Zogby had it at 14/70. On January 26th, it was 36/45. In early March it was 40/48. The poll he put out yesterday had it at 45/47. Doesn't this (steadily improving) number itself reflect well on Obama?


Zogby has one goal in mind with these weak online polls - bookings on FoxNews and a headline on Drudge. Poor Zogby - I don't even think Drudge paid attention to this one.

____________________

hobetoo:

I think Zogby didn't get traction for two reasons: (a) he was truly an outlier, and (b) only the RW media paid much attention. It could also be that Silver's early debunking -- and reminder of Zogby's incompetence -- helped in dampening interest in the Zogby "story."

In any case there was a lot more news interest in other issues -- including Obama's planned presser -- than this poll by a somewhat discredited pollster.

____________________

William:

I think we should not be reacting to a voluntary sample survey, regardless of who conducted it. This information is likely to be skewed due to participation bias.

____________________

Bigmike:

You get no arguments from me. Zogby is obviously getting different numbers.

Where was this same analysis last fall when CBS and the Daily Kos were in the tank for Obama? I must have missed that graph.

Overall, I actually do like this site. There is lots of good info in an easy to understand format. But, like Zogby, take it with a grain of salt.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR