Pollster.com

June 1, 2007

POLL: IVR Polls Florida Democratic Primary

By Eric Dienstfrey

A new automated survey (via MyDD) from the company IVR Polls of registered Democrats who regularly vote in general and Democratic primary elections in Florida (conducted 5/31) finds:

• Among 487 registered Democrats, Sen. Hillary Clinton (at 45%) leads Sen. Barack Obama (18%) and former Sen. John Edwards (14%) in a Democratic primary.

• Among 259 registered Democrats who regularly vote in Democratic primary elections, Clinton (at 51%) leads Edwards (17%) and Obama (11%).

Note: The survey was based on a registration based sample (RBS) that used actual vote history to identify regular primary voters.

By Eric Dienstfrey on June 1, 2007 9:39 PM | Permalink | Comments (4) | TrackBacks (0)

POLL: Rasmussen Party ID

By Eric Dienstfrey

Rasmussen Reports has released their party identification summary for 15,000 interviews conducted in May. "For the fourth month straight, the number of those identifying themselves as Republicans has decreased" (now 31%); those who identify themselves as Democrats has also decreased (now 36%).

By Eric Dienstfrey on June 1, 2007 8:32 PM | Permalink | Comments (0) | TrackBacks (0)

Immigration: Paying Attention?

By Mark Blumenthal

In a post on Wednesday on the recent CBS News/New York Times poll on immigration, I argued that survey respondents will often answer questions about complex public policy issues for which they lack pre-existing opinions. Respondents frequently form on-the-spot opinions, drawing on other previously held attitudes or values cued by the question text. To illustrate that point, I used a cross-tabulation from a recent SurveyUSA automated poll in Charleston, South Carolina and in so doing, inadvertently stumbled into a more complicated question: Whether the relatively small number that are closely following news about the immigration reform bill may be more or less supportive of the bill than other Americans.

The intriguing discrepancy at the heart of both issues is that while majorities of Americans react positively to the "the major provisions" of the immigration reform bill as described by the CBS/New York Times poll, the Rasmussen automated survey shows a two-to-one (48% to 26%) plurality of likely voters opposed to something described only as the "immigration reform proposal agreed to last week." A recent SurveyUSA sampling of adults in the Charleston, South Carolina media market showed a similar result.

I argued -- and continue to believe --that the vast majority of Americans have little sense of the details of the immigration reform bill. Thus, I speculated on Wednesday that the negative reaction measured by the automated pollsters is mostly a reflection of Americans' underlying attitudes toward illegal immigration (most say it is a big problem) and the way the government seems to be handling the issue (most say badly).

To support that speculation I offered a crosstabulation from the SurveyUSA Charleston study that showed lower levels of support for "the immigration reform bill" among those who said that they did not understand the provisions of the bill well.

The problem is that in a release I had not yet seen, Rasmussen Reports provided a similar tabulation showing essentially the opposite result. While all voters in his survey oppose the immigration bill by a 48% to 26% margin, the 37% that say they are following news about the bill "very closely" oppose the legislation by an even bigger margin (69% to 23%). While Rasmussen did not include the complete cross-tabulation, we can extrapolate that the less attentive respondents divided more closely, with 28% in favor, 32% opposed and 40% unsure.

So we have another puzzle. One possibility, of course, is that adults in the Charleston, SC media market have a different perspective on immigration than adults nationally. But a more convincing explanation comes via email from Mickey Kaus, who argues that saying you "understand the provisions" of the bill is different from saying you are "following news stories" about it: "Neither accurately captures whether someone does or does not understand the bill. In fact, the more you follow it the more you realize you don't understand it."

So for both reasons, let's set aside that SurveyUSA result. But that leaves us to consider the more complicated question I inadvertently stumbled into: Does paying more attention to news about the immigration debate make you more opposed to the bill? It might, but unfortunately that question is virtually impossible to resolve with the data available because -- as any good researcher will tell you -- correlation is not causation. If the most attentive tend to be more negative about the immigration bill, that may be because their greater exposure soured them or because those who pay more attention were more inclined to oppose an "immigration reform" bill from the beginning.

The CBS/New York Times data provide a hint that this pattern may be the results of the latter phenomenon. A tabulation provided to Pollster.com by Kathy Frankovic, the polling director at CBS, shows that those who have heard or read "a lot" about "changing the laws of immigration" are significantly more likely to agree that "our immigration policy has so much wrong with it that we need to completely rebuild it" (56%) than those who have heard "some" (47%) or "not much/nothing at all" (46%):

Even this result provides only a clue, and complicating it all further, the Rasmussen and CBS/NYT polls are about as different as two surveys can be (including mode, population, field dates and question wording, to name the most obvious). Without a news exposure experiment or at least a set of surveys that tracks changing attitudes over time, we really cannot know for certain either way.

Before moving on, I want to take this discussion back from the minutia: The polling results worth watching most closely are those that measure pre-existing attitudes about immigration in general, illegal immigration in particular and the job the government seems to be doing handling both (but especially the flood of illegal immigrants). These questions -- as well as those that probe reactions to words like "amnesty" and "deportation" -- will provide the best guide to the way public opinion on this issue will drive both the legislative debate and the upcoming presidential campaign.

PS: Kathy Frankovic's column this week is also deals with the immigration poll and focuses on the unique way that attitudes on this subject cross party lines.

By Mark Blumenthal on June 1, 2007 7:28 PM | Permalink | Comments (0) | TrackBacks (0)

POLL: Gallup Poll Chances of Winning

By Eric Dienstfrey

A new Gallup Panel national survey of 1,007 adults (conducted 5/21 through 5/24) finds:

• 83% think Sen. Hillary Clinton has an "excellent" or "good" chance of winning the Democratic party's nomination; 77% say the same about Sen. Barack Obama, 53% about former Sen. John Edwards, 35% about former V.P. Al Gore.

• 76% think former Mayor Rudy Giuliani has an "excellent" or "good" chance of winning the Republican nomination; 60% say the same about Sen. John McCain, 42% about former Gov. Mitt Romney, 32% about former Sen. Fred Thompson, 28% about for former Speaker Newt Gingrich.

By Eric Dienstfrey on June 1, 2007 3:32 PM | Permalink | Comments (0) | TrackBacks (0)

POLL: Winthrop/ETV South Carolina Primary

By Eric Dienstfrey

A new Winthrop/ETV statewide survey of 670 registered voters in South Carolina (conducted 5/16 through 5/27) finds:

• Among Democrats, Sen. Hillary Clinton leads Sen. Barack Obama (29% to 21%) in a statewide primary; former Sen. John Edwards trails at 11%.

• Among Republicans, former Mayor Rudy Giuliani runs at 19%, Sen. John McCain at 14%, and former Gov. Mitt Romney at 12% in a statewide primary.

• 84% (90% of Democrats, 78% of Republicans) believe a woman "should be able to obtain a legal abortion in certain circumstances."

By Eric Dienstfrey on June 1, 2007 12:04 AM | Permalink | Comments (0) | TrackBacks (0)

May 31, 2007

NYT: Online Polls and Traditional Pollsters

By Mark Blumenthal

Let's start, once again, with the unavoidable conflict: Pollster.com is owned and sponsored by Polimetrix, a company that conducts online surveys. We are, however, walled off in a way that allows us the editorial freedom to write whatever we choose to write (about online surveys and anything else), while also keeping us ignorant of the work that Polimetrix does for its clients.

Case in point is the must-read article on online surveys that appears in today's New York Times. I heard about it not from my corporate overlords, but from a colleague who posted it on the member-only listserv of the American Association for Public Opinion Research. The key quote:

Despite the strong skepticism, Internet-based survey results are likely to get some publicity during the 2008 elections, and executives from companies that conduct these surveys hope that they can use the attention to gain credibility for their methods.

YouGov, for example, has formed a partnership with Polimetrix, an online survey company based in Palo Alto, Calif., for surveys in the United States. Polimetrix, with a panel of one million people, plans to track the 2008 presidential election with a 50-state survey covering a minimum of 1,000 panelists in each state.

"State-by-state election results are an important way for us to prove that our methodology delivers accurate results," said Douglas Rivers, a Stanford University political science professor who founded Polimetrix in 2004. "You can be lucky once, but not 50 times."

Professor Rivers said that the margin of error for Polimetrix surveys is similar to that of polls conducted by telephone. YouGov said that its own results in recent British elections were as close or closer to the actual votes than traditional polling methods.

Oh, so many conflicts, here. I also serve on the executive committee of the American Association for Public Opinion Research (AAPOR), which has condemned as "misleading" the reporting of a margin of sampling error associated with opt-in panels. I supported that statement and, as regular readers know, have been critical of online surveys that report a "margin of error. So on what basis can Rivers claim that "the margin of error for Polimetrix surveys is similar to that of polls conducted by telephone?"

Leaping at the opportunity to bite the hand that feeds me, I emailed Doug Rivers for his comment. His response on the narrower issue of sampling error is a bit technical, and I have copied it below. His larger argument is about how all surveys deal with bias. Telephone surveys that begin with random samples of working telephone numbers suffer some bias in representing all adults due to those who lack landline phone service (coverage) or refused to participate in the survey (non-response). We know, for example that they tend to under-represent younger Americans and those who live in more urban areas. To try to reduce or eliminate this bias, pollsters currently weight by demographic variables (such as age, race, education level and urban/rural geography).

As a pool of potential respondents "opt-in" Internet panels also suffer a bias -- how big is also a matter of some debate -- because panel members must have Internet access, discover the survey panel (usually through an advertisement on a web site) and volunteer to participate. Rivers believes his "sample matching" technique will ultimately do a better job reducing the bias of the opt-in panel universe than standard weighting does to reduce the bias in standard telephone surveys. That is the crux of his argument.

Like a lot of my colleagues, I remain skeptical, but nonetheless committed to an empirical evaluation. As I wrote in Public Opinion Quarterly in 2005 (well before I had any business relationship with Polimetrix):

At what point, if ever, might we place greater trust in surveys drawn from opt-in panels? The only way we will know is by continued experimentation, disclosure, and attempts to evaluate the results through the Total Survey Error framework. Opt-in panels are gaining popularity, whether we approve or not. We should encourage those who procure and consume such research to do so with great caution and to demand full disclosure of methods and results. If nonprobability sampling can ever routinely deliver results empirically proven more valid or reliable, we will need to understand what produces such a result.

In a few days, Charles Franklin and I will begin posting the findings we presented at the AAPOR conference, which include an assessment of how the Polimetrix and other panel surveys did in 2006. This is obviously a debate that will continue in the world of survey research, and we will try to follow along, conflicts and all.

PS: Doug Rivers response regarding the "margin of error:"

Standard errors measure sampling variability & sampling variability is easy to calculate when observations are independent, which they definitely are for large opt-in panels and phone surveys. (The standard error calculation is more complicated for cluster samples, where the observations aren't independent, but these samples aren't clustered; matching introduces another source of variability, but the effect on standard errors is relatively small.) The MSE [mean squared error] calculations involve squared bias and my ambition is to beat your average phone survey in MSE by bias reduction.

By Mark Blumenthal on May 31, 2007 10:33 PM | Permalink | Comments (0) | TrackBacks (0)