Pollster.com

Articles and Analysis

 

More on the Newsweek Poll

Topics: 2008 , Disclosure , The 2008 Race

Family obligations and a nagging cold virus kept me mostly off the grid this weekend while the blogs were abuzz over the latest Newsweek poll of Iowa likely caucus goers. So while late, let me add a few thoughts to those already offered over elsewhere.

First, the margins of sampling error reported by Newsweek -- +/- 7% for the likely Democratic caucus goers and +/- 9% for he Republicans - means that statistically meaningful conclusions are all but impossible regarding Barack Obama's "slight edge" (28% to 24%) over Hillary Clinton. Strictly speaking, even Mitt Romney's 9 point advantage does not attain the usual 95% confidence level that pollsters require to describe a lead as "statistically significant."

Noam Scheiber wonders about what the pollsters could say about the probability of an Obama lead among likely caucus goers, if not 95%. My best guess (assuming that the reported margins of error were based on the usual 95% confidence level) is that the probability of an Obama lead based on the Newsweek poll is about 50%. In other words, the odds of Obama "leading" on this poll are no better than a coin-flip, if we were to take repeated samplings of exactly the same design.

But Matt Yglesias makes the more important point:

It seems to me that there's no real point in arguing about the significance of the rather large +/- 7 points margin of error on this Newsweek poll . . . For something like this, uncertainty about the likely voter screen are probably going to be a bigger problem than sampling error anyway.

He is exactly right. Since July we have seen 12 public polls released in Iowa by 9 different organizations, and each appears to define and sample the likely caucus-goer universe differently. To the extent that pollsters have revealed the details, their snapshots of the electorate are poles apart, to say nothing of the candidates that those voters support. A month ago, for example, I found the percentage of first-time caucus-goers reported on four different polls of Democrats varying from 3% to 43%, with Edwards doing worse (and Clinton better) as the percentage of newcomers increased. The Newsweek survey reports 36% of likely Democratic caucus goers saying "this would be your first caucus."

Unfortunately, the Newsweek release omits many of the same methodological details left out of the other Iowa polling releases (including, remarkably enough, the number of interviews conducted with likely Democratic and likely Republican caucus goers). I have emailed Newsweek's pollsters the same questions we sent last week to the other Iowa pollsters and will include their responses when we begin reporting on the Disclosure Project

By the way, Yglesias also makes another important point: In a truly close race, the ultimate winner among the Democrats may depend on the second choices forced by the convoluted Caucus rules on those whose first choice fails to achieve "viability" (usually 15% of the vote) in their precinct. Remember that the official results for the Democrats will not be a head-count of the first preference of all caucus goers (as in a poll) but rather the estimated share of state delegates won by each candidate based on the final choices at the end of the night. So even if pollsters agreed on how to sample "likely caucus goers," the numbers would still be inconclusive in a close race.

Update:: Slate's Christopher Beam, who called just before I wrote this item, has more

 

Comments
Jeff Winchell:

So why don't pollsters in Iowa attempt to better simulate this precinct-level 15% threshold problem? It seems to me that if you don't do that, the poll is meaningless, even if you could do a reasonable "likely caucus attendee" sampling.

Do any of the Iowa polls even give the voter a 2nd choice? If so, could we see how the numbers work out there?

Or would the sample sizes have to be so large for each precinct, that the only statistically valid sample size approximates the number of actual caucus attendees?

____________________

Mark Blumenthal:

Jeff:

You can at least look to see whether the second choices of voters supporting candidates who get less than 15% skew the standings when reallocated. The Pew Research Center did this in a 2003 survey, but saw no reshuffling. Newsweek asked a second choice question on this survey and could do the same.

The problem with that sort of analysis is two-fold. First, the real thing happens at the precinct level, not statewide, so regional differences are important and practically impossible for a poll to model. Second, a lot of the "tactical voting" is driven by the campaigns rather than voter preferences (e.g. the last minute 2004 "deal" between Kucinich and Edwards).

By the way, a bit of perspective: Notice that the first and second place candidates in that early December Pew Iowa poll (Dean and Gephardt) finished 3rd and 4th respectively.

____________________

Anonymous:

What they should do is limit the polling to those who have actually attended previous caucuses. The caucus process is intimidating and statistically there never are a lot of first timers. Most caucusgoers are older and have caucused many times.

So just sample them. Thats easier than trying to differentiate between "democratic voters" and "likely caucus goers" There are probably some people polled as "Obama supporters" who are really revved up about him and say they are going to caucus. But then nobody tells them they have to actually "re-register" as democrats and they never show up.

This happened with Dean four years ago. Dean's Iowa poll numbers were always inflated with support from people who were never going to actually caucus, such as 18 or 19 year olds who won't vote when it comes down to it.

So just limit to previous caucus goers only and you'll get the real sample.

____________________

TNDem:

As accurate polling in the IA system using standard methodology is difficut, it may be necessary to utilize two separate methods. Perfom one analyis using all registered Democrats to get a sense of the state of the race. Then acquire data from an independent sample of former caucus goers and contrast these findings. But again, I have to argue against this notion that polls are meaningless. The interpretation of polls, as long as the methodology is sound, is always subjective and primarily serves a purpose for pundits, fundraisers and those who would provide endorsements. The question of interest is accuracy in terms of predictive validity. This poll is obviously meaningful or it would not have generated the level of parsing and hysteria that has happened so far among certain campaigns. Whether it have concurrent or predictive validity, well, will just have to see.

____________________

Anonymous:

Can't you work backwards from the margin of error to get an approximate number of people surveyed for each sample?

Using this one, I calculated:

196 likely caucus-going democrats
384 total democrats

119 likely caucus-going republicans
267 total republicans

Obviously this isn't exact for a lot of reasons, but it looks like their screen is about 40-50% attending their caucus, which is probably about twice or three times reality.

____________________

I think it's time to replace Gore in your sensitivity chart with Richardson.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR