Pollster.com

Articles and Analysis

 

'Malpractice?' Afraid So

Topics: John Sides , Quinnipiac University Poll , Sampling Error

As the third reference on Pollster in less than 24 hours, this post will amount to piling on, but I think it needs to be said: John Sides is right. The headline and lead on yesterday's Quinnipiac University poll does amount to "malpractice."

Here's the background. Yesterday, Quinnipiac released results from a new national survey of registered voters with the following headline:

April 21, 2010 - Obama's Bounce Goes Flat, Quinnipiac University National Poll Finds; But Voters Confident He Will Pick Good Judge.

Their lead paragraph explained:

President Barack Obama's job approval, which bounced slightly to a 45 - 46 percent split March 25 in the wake of his health care victory, has flattened out at 44 - 46 percent, his lowest approval rating since his inauguration, according to a Quinnipiac University national poll released today.

Like Sides, I have no grudge against Quinnipiac. They have always been exceptionally responsive and willing to disclose additional results and methodological details on request. Just yesterday polling director Doug Schwartz kindly responded to my email while on vacation to direct an assistant to respond to my request for party ID numbers in Florida. But the "Bounce Goes Flat" lead is just plain wrong. Here are Obama's job approval numbers as measured by Quinnipiac since January:

2010-04-22-quinnipiac obama job.png

I'll let Sides explain:

There is no "bounce." There is no "flattening out." There is nothing but a big fat flat line. It doesn't make for a good press release, but that's the truth.

I'll be the first to concede that when writing about poll numbers it's easy to slip and describe small, non-significant differences as meaningful. I've certainly fallen into that trap myself from time to time. How many times can you write "not quite statistically significant" before readers' eyes start to glaze over? You can use "nominal" to modify "difference," but I'm guessing that few understand the implied technical meaning.

The headline and lead paragraph of a public polling release are something altogether different, especially when written by pollsters. After all, it's our job to get this stuff right. If we can't, how can we complain about the journalists and pundits who take our conclusions and run with them. Cases in point from yesterday:

Time's Mark Halperin: "Obama's Approval Flattens Out."

The Wall Street Journal's Washington Wire: "Just after signing the health bill, Obama's job approval rating ticked up, with 45% of voters approving and 46% disapproving. But the latest poll shows Obama's approval rating hasn't budged and now stands at the lowest level since he took office: 44% approve, 46% disapprove."

Daniel Foster on National Review's The Corner: "From Bounce to Dribble"

National Review's Jim Geraghty: "Quinnipiac: Health-Care Passage Moves Obama's Approval From 45 . . . to 44. Hey, look, it's the health-care-passage bounce! Nah, just kidding."

Those references leave out the scores of bloggers that simply quoted the entire lead sentence or paragraph.

If pollsters can get this sort of thing right, who will?

 

Comments
Westwoodnc Westwoodnc:

Q is just being snarky. After the Demos jubilant predictions that after Obamacare, Obama will end up in Mt. Rushmore, Q basically said "Yeah, here's the 'bounce', it's just a little flat."

____________________

GARY WAGNER:

Oh how quickly the favorite messengers fall from favor when they are bearers of bad news.

Liberals have been in love with Quinnipiac for years because it almost always showed the most favorable results for democrats as the others polls.

Now that they have realeased back to back polls that show that people aren't in love with Obama as much as the liberals, they are chewing up and spitting out Quinnipiac.

I see the bounce in those numbers. Its a small one, but it's there.

____________________

Aaron_in_TX:

The only bounce seemed to occur in the "don't know" numbers.

____________________

sfcpoll:

Forget piling on, what is going on at Quinnipiac? Both a misleading headline on their domestic release AND shoddy double-negative question wording on their nuclear weapons questions- looking a little bit shady.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR