May 27, 2007 - June 2, 2007


POLL: IVR Polls Florida Democratic Primary

A new automated survey (via MyDD) from the company IVR Polls of registered Democrats who regularly vote in general and Democratic primary elections in Florida (conducted 5/31) finds:

  • Among 487 registered Democrats, Sen. Hillary Clinton (at 45%) leads Sen. Barack Obama (18%) and former Sen. John Edwards (14%) in a Democratic primary.

  • Among 259 registered Democrats who regularly vote in Democratic primary elections, Clinton (at 51%) leads Edwards (17%) and Obama (11%).

Note: The survey was based on a registration based sample (RBS) that used actual vote history to identify regular primary voters.

POLL: Rasmussen Party ID

Rasmussen Reports has released their party identification summary for 15,000 interviews conducted in May. "For the fourth month straight, the number of those identifying themselves as Republicans has decreased" (now 31%); those who identify themselves as Democrats has also decreased (now 36%).

Immigration: Paying Attention?

In a post on Wednesday on the recent CBS News/New York Times poll on immigration, I argued that survey respondents will often answer questions about complex public policy issues for which they lack pre-existing opinions. Respondents frequently form on-the-spot opinions, drawing on other previously held attitudes or values cued by the question text. To illustrate that point, I used a cross-tabulation from a recent SurveyUSA automated poll in Charleston, South Carolina and in so doing, inadvertently stumbled into a more complicated question: Whether the relatively small number that are closely following news about the immigration reform bill may be more or less supportive of the bill than other Americans.

The intriguing discrepancy at the heart of both issues is that while majorities of Americans react positively to the "the major provisions" of the immigration reform bill as described by the CBS/New York Times poll, the Rasmussen automated survey shows a two-to-one (48% to 26%) plurality of likely voters opposed to something described only as the "immigration reform proposal agreed to last week." A recent SurveyUSA sampling of adults in the Charleston, South Carolina media market showed a similar result.

I argued -- and continue to believe --that the vast majority of Americans have little sense of the details of the immigration reform bill. Thus, I speculated on Wednesday that the negative reaction measured by the automated pollsters is mostly a reflection of Americans' underlying attitudes toward illegal immigration (most say it is a big problem) and the way the government seems to be handling the issue (most say badly).

To support that speculation I offered a crosstabulation from the SurveyUSA Charleston study that showed lower levels of support for "the immigration reform bill" among those who said that they did not understand the provisions of the bill well.


The problem is that in a release I had not yet seen, Rasmussen Reports provided a similar tabulation showing essentially the opposite result. While all voters in his survey oppose the immigration bill by a 48% to 26% margin, the 37% that say they are following news about the bill "very closely" oppose the legislation by an even bigger margin (69% to 23%). While Rasmussen did not include the complete cross-tabulation, we can extrapolate that the less attentive respondents divided more closely, with 28% in favor, 32% opposed and 40% unsure.

So we have another puzzle. One possibility, of course, is that adults in the Charleston, SC media market have a different perspective on immigration than adults nationally. But a more convincing explanation comes via email from Mickey Kaus, who argues that saying you "understand the provisions" of the bill is different from saying you are "following news stories" about it: "Neither accurately captures whether someone does or does not understand the bill. In fact, the more you follow it the more you realize you don't understand it."

So for both reasons, let's set aside that SurveyUSA result. But that leaves us to consider the more complicated question I inadvertently stumbled into: Does paying more attention to news about the immigration debate make you more opposed to the bill? It might, but unfortunately that question is virtually impossible to resolve with the data available because -- as any good researcher will tell you -- correlation is not causation. If the most attentive tend to be more negative about the immigration bill, that may be because their greater exposure soured them or because those who pay more attention were more inclined to oppose an "immigration reform" bill from the beginning.

The CBS/New York Times data provide a hint that this pattern may be the results of the latter phenomenon. A tabulation provided to Pollster.com by Kathy Frankovic, the polling director at CBS, shows that those who have heard or read "a lot" about "changing the laws of immigration" are significantly more likely to agree that "our immigration policy has so much wrong with it that we need to completely rebuild it" (56%) than those who have heard "some" (47%) or "not much/nothing at all" (46%):


Even this result provides only a clue, and complicating it all further, the Rasmussen and CBS/NYT polls are about as different as two surveys can be (including mode, population, field dates and question wording, to name the most obvious). Without a news exposure experiment or at least a set of surveys that tracks changing attitudes over time, we really cannot know for certain either way.

Before moving on, I want to take this discussion back from the minutia: The polling results worth watching most closely are those that measure pre-existing attitudes about immigration in general, illegal immigration in particular and the job the government seems to be doing handling both (but especially the flood of illegal immigrants). These questions -- as well as those that probe reactions to words like "amnesty" and "deportation" -- will provide the best guide to the way public opinion on this issue will drive both the legislative debate and the upcoming presidential campaign.

PS: Kathy Frankovic's column this week is also deals with the immigration poll and focuses on the unique way that attitudes on this subject cross party lines.

POLL: Gallup Poll Chances of Winning

A new Gallup Panel national survey of 1,007 adults (conducted 5/21 through 5/24) finds:

  • 83% think Sen. Hillary Clinton has an "excellent" or "good" chance of winning the Democratic party's nomination; 77% say the same about Sen. Barack Obama, 53% about former Sen. John Edwards, 35% about former V.P. Al Gore.

  • 76% think former Mayor Rudy Giuliani has an "excellent" or "good" chance of winning the Republican nomination; 60% say the same about Sen. John McCain, 42% about former Gov. Mitt Romney, 32% about former Sen. Fred Thompson, 28% about for former Speaker Newt Gingrich.

POLL: Winthrop/ETV South Carolina Primary

A new Winthrop/ETV statewide survey of 670 registered voters in South Carolina (conducted 5/16 through 5/27) finds:

  • Among Democrats, Sen. Hillary Clinton leads Sen. Barack Obama (29% to 21%) in a statewide primary; former Sen. John Edwards trails at 11%.

  • Among Republicans, former Mayor Rudy Giuliani runs at 19%, Sen. John McCain at 14%, and former Gov. Mitt Romney at 12% in a statewide primary.

  • 84% (90% of Democrats, 78% of Republicans) believe a woman "should be able to obtain a legal abortion in certain circumstances."

NYT: Online Polls and Traditional Pollsters

Topics: Internet Polls , Pollster.com

Let's start, once again, with the unavoidable conflict: Pollster.com is owned and sponsored by Polimetrix, a company that conducts online surveys. We are, however, walled off in a way that allows us the editorial freedom to write whatever we choose to write (about online surveys and anything else), while also keeping us ignorant of the work that Polimetrix does for its clients.

Case in point is the must-read article on online surveys that appears in today's New York Times. I heard about it not from my corporate overlords, but from a colleague who posted it on the member-only listserv of the American Association for Public Opinion Research. The key quote:

Despite the strong skepticism, Internet-based survey results are likely to get some publicity during the 2008 elections, and executives from companies that conduct these surveys hope that they can use the attention to gain credibility for their methods.

YouGov, for example, has formed a partnership with Polimetrix, an online survey company based in Palo Alto, Calif., for surveys in the United States. Polimetrix, with a panel of one million people, plans to track the 2008 presidential election with a 50-state survey covering a minimum of 1,000 panelists in each state.

"State-by-state election results are an important way for us to prove that our methodology delivers accurate results," said Douglas Rivers, a Stanford University political science professor who founded Polimetrix in 2004. "You can be lucky once, but not 50 times."

Professor Rivers said that the margin of error for Polimetrix surveys is similar to that of polls conducted by telephone. YouGov said that its own results in recent British elections were as close or closer to the actual votes than traditional polling methods.

Oh, so many conflicts, here. I also serve on the executive committee of the American Association for Public Opinion Research (AAPOR), which has condemned as "misleading" the reporting of a margin of sampling error associated with opt-in panels. I supported that statement and, as regular readers know, have been critical of online surveys that report a "margin of error. So on what basis can Rivers claim that "the margin of error for Polimetrix surveys is similar to that of polls conducted by telephone?"

Leaping at the opportunity to bite the hand that feeds me, I emailed Doug Rivers for his comment. His response on the narrower issue of sampling error is a bit technical, and I have copied it below. His larger argument is about how all surveys deal with bias. Telephone surveys that begin with random samples of working telephone numbers suffer some bias in representing all adults due to those who lack landline phone service (coverage) or refused to participate in the survey (non-response). We know, for example that they tend to under-represent younger Americans and those who live in more urban areas. To try to reduce or eliminate this bias, pollsters currently weight by demographic variables (such as age, race, education level and urban/rural geography).

As a pool of potential respondents "opt-in" Internet panels also suffer a bias -- how big is also a matter of some debate -- because panel members must have Internet access, discover the survey panel (usually through an advertisement on a web site) and volunteer to participate. Rivers believes his "sample matching" technique will ultimately do a better job reducing the bias of the opt-in panel universe than standard weighting does to reduce the bias in standard telephone surveys. That is the crux of his argument.

Like a lot of my colleagues, I remain skeptical, but nonetheless committed to an empirical evaluation. As I wrote in Public Opinion Quarterly in 2005 (well before I had any business relationship with Polimetrix):

At what point, if ever, might we place greater trust in surveys drawn from opt-in panels? The only way we will know is by continued experimentation, disclosure, and attempts to evaluate the results through the Total Survey Error framework. Opt-in panels are gaining popularity, whether we approve or not. We should encourage those who procure and consume such research to do so with great caution and to demand full disclosure of methods and results. If nonprobability sampling can ever routinely deliver results empirically proven more valid or reliable, we will need to understand what produces such a result.

In a few days, Charles Franklin and I will begin posting the findings we presented at the AAPOR conference, which include an assessment of how the Polimetrix and other panel surveys did in 2006. This is obviously a debate that will continue in the world of survey research, and we will try to follow along, conflicts and all.

PS: Doug Rivers response regarding the "margin of error:"

Standard errors measure sampling variability & sampling variability is easy to calculate when observations are independent, which they definitely are for large opt-in panels and phone surveys. (The standard error calculation is more complicated for cluster samples, where the observations aren't independent, but these samples aren't clustered; matching introduces another source of variability, but the effect on standard errors is relatively small.) The MSE [mean squared error] calculations involve squared bias and my ambition is to beat your average phone survey in MSE by bias reduction.

Support for the Death Penalty and Question Wording

Topics: Divergent Polls


One great value of long term data archives for polling information is the ability to compare trends over long spans of time and across different question wording. The Roper Center at the University of Connecticut provides one of these archives. Its subscription only iPoll database includes over 500,000 questions that pollsters have asked since 1935. This is a uniquely valuable resource for opinion research.

Public support for the death penalty over time is a good example of the power of this collection. The plot above looks at death penalty support and opposition since 1936 in Gallup polls. Gallup has maintained a constant question wording over that span giving us the best long term look at this issue. Gallup has a summary of these data on a no-subscription required page here. It provides more data than I look at here.

My point today is two-fold. First, just to look at the trends over time. The standard Gallup question plotted above is

    Are you in favor of the death penalty for a person convicted of murder?

Support for the death penalty declined to a low in 1960-75, with support around 50% and opposition around 40% (with some bouncing around a bit.) Following 1975 there was a long growth in support for the death penalty, with support rising to as high as 80% and opposition falling into the teens.

Since the decline in homicide rates in the 1990s, support has declined by about 10 percentage points since 2000, to around 70%, with opposition rising to the mid-20s.

The public overwhelmingly supports "the death penalty for a person convicted of murder."

But now let's consider what happens when we offer a specific alternative to the death penalty. In the classic Gallup question, there is no explicit alternative to the death penalty, so respondents must imagine for themselves what the alternative might be.

Since 1985, Gallup has also asked an alternative question which includes a specific alternative sentence:

    If you could choose between the following two approaches, which do you think is the better penalty for murder--the death penalty or life imprisonment, with absolutely no possibility of parole?

This reflects a shift in the policy debate, as no-parole sentences were raised as an alternative to execution.

Presented with this alternative, opinion shifts substantially:


With the "life without parole" option, support for execution falls to just over 50%, while support for life terms rises to the mid 40s. Still a majority in favor of the death penalty, but a substantially more closely divided public than with the classic question.

Two points from this. First, questions define the alternatives respondents are encouraged to consider. The classic question offers one option and leaves it to the respondent to imagine others. The death vs life introduces an explicit alternative, and finds much more support for a penalty short of death. However, we could imagine a question with a third option: life but with a possible parole. Or even specific sentence lengths (10 years? 20? 50?). In part the question reflects the policy debate. But survey questions must also necessarily limit the range of options under consideration. In this, there can be no escape from question wording effects, and no end to argument about whether questions "really" capture all the issues. The fact is every question includes some and excludes other options.

The second point is the political one. If one opposed the death penalty, one might be more likely to find support by arguing forcefully for hard time and no parole rather than arguing against the death penalty in principle.

Cross-posted at Political Arithmetik.

POLL: Siena NY Primary

A new Siena Research Institute statewide survey of 620 registered voters in New York State (conducted 5/18 through 5/25) finds:

  • Among 301 Democrats, Sen. Hillary Clinton (at 42%) leads Sen. Barack Obama and former V.P. Al Gore (both at 13%) in a statewide primary; former Sen. John Edwards trails at 7%.

  • Among 174 Republicans, former Mayor Rudy Giuliani (at 52%) leads Sen. John McCain (14%) and former Gov. Mitt Romney (7%) in a statewide primary. When asked to chose from a list of five candidates, Giuliani runs at 50%, McCain at 12%, former Sen. Fred Thompson at 8%, and Romney and former Speaker Newt Gingrich both at 7%.

  • General election match-ups:

    Clinton 52%, Giuliani 39%
    Clinton 54%, McCain 36%
    Clinton 57%, Thompson 29%
    Obama 50%, Giuliani 40%
    Obama 50%, McCain 33%
    Gore 56%, Thompson 28%

POLL: Quinnipiac Pennsylvania Survey

A new Quinnipiac University statewide survey (08 results, issues results) of 1,318 registered voters in Pennsylvania (conducted 5/22 through 5/28) finds:

  • Among 585 Democrats, Sen. Hillary Clinton (at 33%) leads former V.P. Al Gore (16%), Sen. Barack Obama (13%), and former Sen. John Edwards (11%) in a statewide primary.

  • Among 575 Republicans, former Mayor Rudy Giuliani (at 28%) leads Sen. John McCain (11%), former Sen. Fred Thompson (10%), former Gov. Mitt Romney (9%), and former Speaker Newt Gingrich (8%) in a statewide primary.

  • General election match-ups:

    Giuliani 47%, Clinton 43%
    Giuliani 45%, Clinton Obama 40%
    Giuliani 44%, Gore 45%
    McCain 43%, Clinton 45%
    McCain 42%, Obama 41%
    McCain 44%, Gore 44%

Immigration Polls: Opinion or Reaction?

Last week's CBS News/New York Times survey (CBS story, results; Times story, results) is a classic example of the way public opinion pollsters will try to learn all they can about one complex but critical public policy issue. The survey focused special attention on immigration, asking 38 questions that probed the issue from almost every angle. Not surprisingly, the results to some of those questions have generated controversy. While some of my pollster colleagues will be quick to dismiss the complaints as partisan grousing, this survey provides some helpful lessons about the limitations of what polls can tell us when asking about subjects with which most Americans are unfamiliar.

I find it helpful to divide the immigration questions on the CBS/New York Times poll into two sections. The first twenty or so items (beginning with #41 in either the CBS or the NYTimes releases) use simple neutral questions to probe the presumably pre-existing attitudes that most Americans have about immigration, legal and illegal. We learn, for example (and quoting from the CBS release):

  • "Americans have a positive impression of immigrants generally," yet "are highly critical of current U.S. immigration policy, especially when it comes to illegal immigration."
  • 75% believe most immigrants are "here illegally"
  • 82% "think the U.S. could be doing more along its borders to keep illegal immigrants from crossing over into the U.S.; just 14% think the U.S. is now doing all it can."

These initial results have stirred little controversy, perhaps because they are not particularly new (in fact, as the Times release shows, most have been asked before), and thus, not particularly newsworthy.

Needless to say, what interests everyone -- including those most critical of the CBS/NY Times poll -- is what Americans think of the immigration reform bill now before Congress. So starting at about item #60, the poll asks for reactions to a series of policy proposals. These questions introduce new and sometimes complex information about the various potential immigration policy alternatives. The results, which lead the coverage by both organizations, show "broad support" for "the major provisions" in the immigration bill now before Congress" (quoting the Times story).

These results -- and especially the wording of the questions involved -- have drawn intense criticism from opponents of the immigration bill (see Kaus, Allahpundit, Mark Krikorian). I will not try to referee the various critiques here, except to point out that the results from the second set of questions on the CBS/New York Times poll are better characterized as a measurement of how Americans react to descriptions of the immigration proposals than a measurement of Americans' current opinions on those proposals.

How much do Americans really know about the immigration reforms currently being debated in Congress? One question from the CBS/NYTimes poll suggests not very much: Only 26 % say they have read or heard "a lot" about "changing the laws regarding immigration in the United States;" 51% have heard "some," 18% not much, 4% nothing at all.

Moreover, when other pollsters have asked about the immigration bill in general terms (without providing specific descriptions), reactions are more negative than they are to the provisions tested on the CBS/NYTimes poll. For example, the Rasmussen Reports automated survey asked: "From what you know about the agreement, do you favor or oppose the immigration reform proposal agreed to last Week?" Roughly half (48%) opposed the agreement, about a quarter (26%) favored it with just as many (26%) unable to answer.

Survey researchers have long understood that respondents will readily answer questions about unfamiliar issues, typically drawing on their real attitudes or values concerning the subject of the question. Americans do have a sense of how their government is handling immigration (badly - see above), and so it is not surprising to see skepticism about whatever Congress is up to from those not following the immigration debate closely (see the analysis by Scott Rasmussen).

Two questions from a recent automated survey conducted in the Charleston, South Carolina media market by SurveyUSA make this point more vividly:

  • Asked if they "understand the provisions" of a "proposed immigration reform bill" resulting from a "deal" reached by "the Senate and the White House," only 10% say they understand it very well, 34% somewhat well, 38% not very well and 17% not at all.
  • 47% say they oppose the "immigration reform bill," 36% support it and 18% are unsure.

Put those two questions together and we see that the less informed respondents are less supportive of the bill. Three out of four who understand little or nothing about the bill offer an opinion, and they oppose the immigration bill by a margin of roughly two to one (49% to 25%). The better informed respondents divide evenly (46% favor, 44% oppose).


So back to the CBS/ New York Times results showing positive reactions to the various specific "provisions" of the immigration reform bill. What should we make of those? Although many respondents are probably forming opinions on the fly in reaction to the text of the questions, their reactions have meaning. They can tell us a lot about how Americans may react, should the news media or the campaigns focus far more attention on the immigration debate than they have thus far. But I would urge a few cautions:

First, it is entirely appropriate to scrutinize the language of the questions as the critics have, and compare and contrast results when pollsters use different language to ask similar questions. Follow the advice of from Frank Newport and my old anonymous friend, Professor M, and consider the various results as the best measure of potential range of support for the legislation.

Second, be realistic about how much more closely Americans will follow the immigration issue and what new information they are most likely to receive. We will certainly hear arguments about whether the immigration bill amounts to "amnesty," but will Americans ever consider it in the level of detail provided by the questions on the CBS/New York Times poll?

Put another way, we might want to think about what similar survey results foretold about previous legislative debates. Consider, for example, the Clinton health care proposal of the early 1990s. In September 1993, on the eve of President Clinton's address to Congress describing his plan, the CBS/New York Times poll tested its widely anticipated elements. Here is how the Robin Toner described the poll results in the New York Times poll story on September 23, 1993 (sub. req.):

The survey shows overwhelming majorities support the idea of assuring health coverage to all Americans and guaranteeing that no one ever loses their insurance when they switch jobs or suffer a medical catastrophe. Sixty-one percent said they were willing to pay higher taxes to achieve those goals, and more than half said they were willing to have the Government require employers to pay most of the health insurance premiums to cover their workers -- a centerpiece of the Clinton plan.

Of course, despite these favorable reactions, the Clinton plan collapsed under an assault by opponents who played to cynicism about government programs, fears of health care rationing and disapproval of the Clinton administration's early missteps. Those underlying, pre-existing attitudes, which were also in evidence in that September 1993 poll, turned out to be the ones that mattered most.

UPDATE: Rasmussen Reports published updated results on immigration this morning (via Kaus). These latest results show a different pattern among the most attentive respondents than what SurveyUSA showed in Charleston:

Overall, despite a major push by the President and others over the past week, support for the Senate bill has not increased at all. In polling conducted last night (Tuesday, May 29), 26% of voters favor passage of the bill. That’s unchanged from the 26% support found in polling conducted the previous Monday and Tuesday. Forty-eight percent (48%) of voters remain opposed.

Eighty-one percent (81%) of American voters are closely following news stories about the issue, including 37% who are following it Very Closely. Those with the highest interest in the issue oppose the legislation by a 3-to-1 margin (69% to 23%). By a 55% to 15% margin, those following the story Very Closely believe the bill will lead to increased levels of illegal immigration.

POLL: Rasmussen Kentucky Gubernatorial

A new Rasmussen Reports automated survey of 500 likely voters in Kentucky (conducted 5/24 through 5/25) finds Steve Beshear (D) leading Gov. Ernie Fletcher (R) 51% to 35% in a gubernatorial general election match-up.

POLL: American Research Group IA/NH/SC

Three new American Research Group statewide surveys of likely primary voters in Iowa, New Hampshire, and South Carolina (conducted 5/23 through 5/25) finds:

  • Among 600 Democrats in each state, Sen. Hillary Clinton leads Sen. John Edwards 34% to 25% 18% in New Hampshire and edges him out by six points in Iowa and by four points in South Carolina; Obama trails with less than 20% in all three states.

  • Among 600 Republicans, Sen. John McCain leads former Mayor Rudy Giuliani 32% to 23% in South Carolina and leads former Gov. Mitt Romney 30% to 23% in New Hampshire. In Iowa McCain edges out Giuliani 25% to 23%.

POLL: Rasmussen 5/29 Round-up

New Rasmussen Reports automated surveys find:

  • Among 559 likely Democratic primary voters, Sen. Hillary Clinton leads Sen. Barack Obama 35% to 26% in a national primary; former Sen. John Edwards trails at 14% (conducted 5/21 through 5/23).

  • Among 469 likely Republican primary voters, former Mayor Rudy Giuliani leads former Gov. Mitt Romney 25% to 26% 16%; Sen. John McCain trails at 15%, former Sen. Fred Thompson at 12% (conducted 5/21 through 5/23).

  • Among 500 likely voters in New York State, Clinton leads Giuliani 50% to 29% in a statewide general election match-up; Mayor Michael Bloomberg trails at 15% (conducted 5/20).

  • Among 500 likely voters in New Jersey, Clinton edges out Giuliani 42% to 39%; Bloomberg trails at 13% (conducted 5/19).

  • Among 500 likely voters in Connecticut, Clinton and Giuliani both run at 40%; Bloomberg trails at 9%.

POLL: Gallup Homosexual Relations

Additional results from Gallup Poll's recent Values and Beliefs survey (analysis, video) of 1,003 adults (conducted 5/10 through 5/13) finds:

  • 59% think homosexual relations between consenting adults should be legal; 37% say it should not.

  • 49% believe homosexual relations are morally wrong; 47% believe they are morally acceptable.

  • 42% believe homosexuality is "something a person is born with;" 35% believe it is "due to factors such as upbringing and environment."

POLL: SurveyUSA Kentucky Gubernatorial GE

A new WCPO-TV/WHAS-TV automated survey of 609 likely voters in Kentucky (conducted 5/23 through 5/24 by SurveyUSA) finds Democrat Steve Beshear leading Republican Gov. Ernie Fletcher 62% to 34% in a gubernatorial general election match-up.