Pollster.com

June 17, 2007 - June 23, 2007

 

Bush Approval: ARG at 27%, Trend at 29.1%

Topics: George Bush

1BushApproval2ndTerm20070621.png

A new American Research Group poll, taken 6/18-21/97, finds approval at 27%, disapproval at 67%. This puts the approval trend estimate at 29.1%, continuing the decline that began in late April.

This is in line with Newsweek's 26% and NBC/WSJ's 29%, and is within range of Gallup's 32%, the three most recent polls. My post on the Newsweek result yesterday is unchanged by this result. None of the recent polls is an outlier, all falling within the +/-5% confidence interval around the current 29.1% estimate.

By popular request, I've added a plot to my standard approval post contents: The fourth graph contains both the standard "old blue" trend estimate, which is more likely to be correct in the long run but which is slower to detect change, and the "ready red" estimator which catches change quickly but is easily mislead by random noise that isn't actually a change in trend. Now you can see both as part of these posts. The blue line is plotted second, so when the red agrees closely with the blue estimator, the red line is covered up by the blue line. So where you can see red at all is where the estimators disagree. Otherwise red and blue are tracking together.

2LastSixPolls20070621.png

3BushResiduals20070621.png

4BushApproval2ndTermRough20070621.png

5BootApproval20070621.png

6Sentitivity2ndTerm20070621.png

Cross-posted at Political Arithmetik.


"Understanding Our Insignificance"


It is Friday afternoon, so -- in the spirit of Friday afternoon cat blogging -- we bring you this spoof from the "Onion News Network." Don't ask, just click:


In The Know: Kim Jong-Il's Approval Rating Plummets to 120


Pew's Keeter: How Serious the Cell Phone Only Problem?

Topics: Cell Phones

During last month's conference of the American Association for Public Opinion Research (AAPOR), I mentioned the many papers being presented on the growth of cell-phone-only households and promised to report back. I have been digging through the many papers ever since, and have been working on some analysis on the subject which, in the crush of new charts and other activity around here, I have yet to post.

Fortunately, Scott Keeter, the director of survey research at the Pew Research Center, has posted a concise and accessible review of the cell phone challenge to surveys. The Pew Center has been at the forefront of research and development on this subject, conducting four pilot studies over the last two years that interviewed people with cell phones over their cell phones.

The summary is well worth reading in full, but for those in a rush, here is Keeter's view of where things are heading:

Pollsters recognize that some type of accommodation for the cell-only population will have to be made eventually, as was clear from the large amount of research on the topic presented at the AAPOR conference last month. In addition to the use of so-called "dual frame samples" such as those described above (calling both a cell phone sample and a landline sample), practitioners are discussing other alternatives, including the establishment of panels of cell-only respondents that can be surveyed periodically to track their opinions, and employing mail or internet surveys to reach the cell-only population.

For those who want more detail, I can also highly recommend the longer paper he presented at the AAPOR conference (co-authored with Courtney Kennedy and April Clark of the Pew Center and Trevor Thompson and Mike Mokrzycki of the Associated Press) which Keeter has now posted online. Of all the papers I have reviewed, it was easily the best review of the issues most relevant to the political surveys we all obsess over.

PS: When I return to the cell phone issue next week, I also want to say more about a question starting to bounce around the Internet (via Sullivan): "Are Cell Phones Killing Ron Paul's campaign."


POLL: Gallup War on Terror


Additional analysis to a recent Gallup national survey of 1,007 adults (conducted 6/11 through 6/14) finds:

  • 29% of Americans (53% of Republicans, 26% of Democrats) believe U.S. and its allies are winning the war on terror -- "the lowest percentage holding this view since the 9/11 terrorist attacks," 20% say the terrorists are, and 50% say neither side is winning.

  • 43% believe the war in Iraq is part of the war on terrorism; 53% say it is a separate military action.


POLL: American Research Group Bush Approval


A new American Research Group national survey of 1,100 adults (conducted 6/18 through 6/21) finds 27% of Americans approving of the job George Bush is doing as president while 67% disapprove, "the highest level of disapproval and lowest level of approval for the Bush presidency."


Bush Approval: Newsweek at 26%; Trend at 29.9%

Topics: George Bush

1BushApproval2ndTerm20070619.png

Newsweek has a new poll taken 6/18-19/07 that finds approval of President Bush at 26%, disapproval at 65%. With this new data point the approval trend estimate stands at 29.9%, the first time the trend has fallen below 30%. The sharpness of the decline is striking. The change-point for approval is April 23, corresponding to the week of the Congressional vote for deadlines and a fund cutoff in Iraq and the President's subsequent veto. It precedes the immigration debate, though that debate may have sustained the decline. (On the other hand there is little evidence that immigration accelerated the decline which was already underway.)

A look at the last six polls is revealing. Newsweek usually has a "house effect" of about 2.2 percentage points below the trend estimate, so finding it below trend is no surprise. But look also at NBC/WSJ. Their house effect has been around -.6, only a shade below trend. But the new NBC/WSJ poll 6/8-11/07 found approval at 29%. And Gallup's house effect is +.55, and their latest reading was 32%. That makes it awfully convincing that approval has now fallen to very nearly 30%, plus or minus 1. (And to reiterate one more time, house effects incorporate many effects that are specific to a given polling organization. These include how don't knows are treated (are people pushed to respond), question wording, sample selection and non-response rates and order of questions within a survey.)

2LastSixPolls20070619.png

Given that agreement among polls, it is not surprising that the outlier analysis finds nothing to complain about. The confidence interval around the trend estimate is approximately +/- 5%, so Newsweek at -3.9% is well within that range.

3BushResiduals20070619.png

The trend over the last 20 estimates has been unrelentingly downward. Despite the long stability in approval from December through mid-April, the trajectory of approval has been dramatically down since April 23.

4Sentitivity2ndTerm20070619.png

Newsweek headlines "How low can he go?" We looked at that a while back in this post. One of the keys I pointed out then was the support presidents enjoy from their own supporters. Past unpopular presidents have suffered a substantial loss of support from their party. President Bush has been a significant exception to this. His lows among Democrats and independents have not been accompanied by similar declines among Republicans. When the post was written in March 2006 approval stood at 38%. Approval among Democrats was 10% back then while independents reported 27% approval. These were much lower than history would predict for a president at 38%. The huge difference was that Republican support in March 2006 was an amazing 82%.

That support has been slipping in recent months. The Newsweek poll finds approval at 6% among Democrats, 23% among independents and 60% among Republicans. The GOP partisans still are providing more support than we might expect, but it is clearly no longer the reservoir of support it once was. (Gallup now finds Democrats at 8%, independents at 24% and Republicans at 73%. Since the historical analysis reported above was based on Gallup polling, a fair comparison would be the shift from 82% in March 2006 to 73% now among Republicans, a less dramatic comparison, but one that still demonstrates the strains on Republican support. It also casts some light on differences between polling organization results.)

Because approval among Democrats is so low, further declines there can make little difference to the overall level. Independents could matter a bit more, but how low approval ultimately goes is going to depend on Republicans' willingness to continue to stand by the president.

Cross-posted at Political Arithmetik.


The Merits of the "Horse Race"

Topics: 2008 , The 2008 Race

This seems to be the week for analytical pieces on the merits of looking at early horse-race polls. In addition to the three I linked to yesterday, add one more today from the New York Times' Robin Toner. So let me take this opportunity to throw in my two cents.

There are many reasons why you might want to look at early polls in the presidential race. Hopefully one of those reasons is not to determine which candidate deserves your support, particularly in the presidential race. Even if picking a winner matters to you, history has shown repeatedly that early front runners sometimes falter, as Toner's piece chronicles in detail. Thus, vote preference questions asked this early in the presidential race "may have very little relationship to the ultimate outcome."

On the other hand, if you are like most candidates, staff, consultants, journalists and ordinary political junkies, your interest more in tracking the progress of the presidential campaigns than in predicting the ultimate winner. If so, you want to know which candidates are succeeding, who is making progress, who is having trouble "getting traction?" If those questions interest you, then the polls you want to follow are the ones in Iowa and New Hampshire. I say that knowing that surveys of potential caucus goers in Iowa face much bigger methodological challenges than those in other contests (and some pollsters rise to those challenges better than others).

Polls in those early states are worth following because of the enormous impact they have had historically on the voters in the primary and caucus states that follow. Consider the advice in the Toner piece from Republican pollster Bill McInturff:

Bill McInturff, a pollster for Mr. McCain in 2000 and today, said that national polls were of limited utility at this stage, and that state polls were subject to drastic change.

"Results in the early races ripple through the primaries," Mr. McInturff said, recalling Mr. McCain's upset victory in New Hampshire seven years ago, just before the race moved to South Carolina. "The day before New Hampshire, McCain was 20 points down in South Carolina. The day after, he was tied."

Moreover, up until January 2008, the candidates will spend the overwhelming majority of their time and cash in Iowa and New Hampshire. They will go elsewhere to raise money, but over the next eight months or so, "the campaign" will mostly occur in Iowa, New Hampshire and a handful of other early states.

The best demonstration is the recent rise of Mitt Romney and Bill Richardson in both Iowa and New Hampshire (see our charts for IA-Rep, NH-Rep, IA-Dem, NH-Dem). The explanation for both trends is easy: Both candidates have been running broadcast television advertising there. Romney has already reportedly spent over $4 million in the two states; Richardson was reportedly buying time at a level that (by my estimate, using last year's rates) looks like about $200K per week (or perhaps total, it's not entirely clear from the article).

Of course, in time, all of the candidates will join the ad war, resulting in some reshuffling of the standings. Who will benefit? Who knows? The safest bet is that by January, most of the candidates will enjoy the same level of name recognition in the early states as the front runners do now. But the point here is that if we want to follow the process of the campaign, the trends in those early states are the ones that matter most, particularly as the information flow to those voters cranks up.

Is there any reason to watch the national horse race numbers now when most voters may not make up their minds in a meaningful way until January 2008? One argument is that we have so much more national data, and the sampling methods are typically more rigorous, consistent and reliable than the approaches used by public polls in early states, particularly in early caucus states like Iowa and Nevada. When we pool national data across multiple surveys -- as we do on the graphs here at Pollster -- the odds are greater that any trends we do see reflect real change. Moreover, some national polls provide unusually rich snapshots of the perceptions of the best known candidates beyond just the horserace preference. The most recent NBC News/Wall Street Journal poll provides a great example (especially the Q16 and Q20 series).

Just keep in mind this warning (via Toner ) from the Pew Research Center's Andy Kohut: "Their attitudes are soft, so tests of their [current] preferences are not reliable." That data back him up even in New Hampshire, as Bruce Reed noticed. More than half of Republicans (57%) and almost as many Democrats (49%) told the CNN/WMUR/UNH poll that they have "no idea" who they will vote for in the presidential primary. Only 8% of New Hampshire Democrats and only 6% of the state's Republicans have "definitely decided who" they will support.


Bloomberg Hurts Giuliani More Than Clinton

Topics: 2008 , The 2008 Race

Bloomberg3way062207.png

Michael Bloomberg's possible entry into the presidential race appears to hurt Giuliani more than Clinton, based on analysis of data from SurveyUSA's recent polling in 16 states. While support for both candidates declines when Bloomberg is included in the vote question, Giuliani's support declines by an estimated 1.7 more percentage points than does Clinton's.

The SurveyUSA data were collected by interactive voice response (i.e. "automated" survey) in AL, CA, IA, KS, KY, MA, MN, MO, NM, NY, OH, OR, TX, VA, WA and WI June 8-10, 2007 with approximately 500 respondents in each state. Respondents were first asked

    If there were an election for President of the United States today, and the only two names on the ballot were Republican Rudy Giuliani and ... Democrat Hillary Clinton, who would you vote for?

The survey then asked

    OK, now imagine that the election for president was a 3 way contest, and the 3 candidates on the ballot are Republican Rudolph Giuliani, Democrat Hillary Clinton, and Independent Michael Bloomberg.

(SurveyUSA asked about other candidate pairs as well, but the focus here is only on the Giuliani-Clinton-Bloomberg effects.)

Adding a third option to any vote question should be expected to draw support away from both candidates in the two candidate only form. In the figure above, that means we would certainly expect the data points to fall below the black 45 degree line in the figure, meaning candidates do worse with three candidates than with two. That obviously occurs in the figure.

The differential impact on the candidates is the more crucial point. If both candidates are equally affected, we'd expec the red and blue points in the figure to mix together more or less randomly in the plot. If one candidate is more damaged than the other, then the blue and red points should separate with one generally closer to the 45 degree line than the other. That's what we see.

Generally the Clinton (blue) points are above the Giuliani (red) points. If we take the simple average changes, Clinton loses an average of 3.6 points when Bloomberg is added, while Giuliani loses an average of 5.2 points. When we do a slightly fancier regression estimate, the net loss hurts Giuliani by 1.7 points more than it does Clinton. That difference is visible in the chart as the gap between the red and blue estimated regression lines.

As a Republican until this week, Bloomberg could be expected to draw more from the Republican than the Democratic candidate.

Of course this is all hypothetical with a mayor who says he expects to serve out his term. But you have to admit that an "all New York" three way race would have its charms.

Cross-posted at Political Arithmetik.


POLL: Newsweek Approvals


A new Newsweek national survey (story, results; released 6/22) finds:

  • 26% approve of the job George Bush is doing as president, "a record low" for Newsweek; 65% disapprove.

  • 25% approve of the way Congress is handling its job; 63% disapprove

** The release ommitted methodological details. We will fill those in once they are available.


POLL: Gallup Congressional Confidence


Additional analysis from a recent Gallup national survey of 1,007 adults (conducted 6/11 through 6/14) finds "[t]he percentage of Americans with a "great deal" or "quite a lot" of confidence in Congress is at 14%, the lowest in Gallup's history of this measure -- and the lowest of any of the 16 institutions tested in this year's Confidence in Institutions survey. It is also one of the lowest confidence ratings for any institution tested over the last three decades."


POLL: ABC/Post & Pew Bloomberg Candidacy


Two new analyses concerning a Mayor Michael Bloomberg presidential run find:

  • ABC's Gary Langer looks at recent polls testing potential independent candidacies. Among the findings, 79% of leaned Democrats and 68% of leaned Republicans are satisfied with their choice of candidates, which, according to Langer, "doesn't seem to leave a whole lot of breathing room for Mayor Mike."

  • Pew's Michael Dimock and Shawn Neidorf look at a recent Pew Research survey that shows 36% of Republicans and 26% of Democrats say there is at least "some chance" they would vote for Bloomberg for president.


POLL: PPP (D) General Election Match-Ups


A new Public Policy Polling (D) automated survey of 545 likely North Carolina voters (conducted 6/19) finds:

  • General election match-ups for President:

    Giuliani 47%, Clinton 43%
    Giuliani 45%, Edwards 46%
    Giuliani 46%, Obama 42%

    McCain 44%, Clinton 45%
    McCain 40%, Edwards 48%
    McCain 45%, Obama 44%

    Romney 41%, Clinton 47%
    Romney 37%, Edwards 51%
    Romney 43%, Obama 47%

    Thompson 46%, Clinton 43%
    Thompson 43%, Edwards 47%
    Thompson 45%, Obama 44%

  • General election match-ups for Senate:

    Dole 46%, Purdue 37%
    Dole 45%, Moore 34%


POLL: Strategic Vision (R) Florida Primary


A new Strategic vision (R) statewide survey of 1,200 likely voters in Florida (conducted 6/15 through 6/17) finds:

  • Among Democrats, Sen. Hillary Clinton (at 37%) leads Sen. Barack Obama (21%) and former Sen. John Edwards (20%) in a statewide primary.

  • Among Republicans, former Mayor Rudy Giuliani runs at 30%, former Sen. Fred Thompson at 24%, Sen. John McCain at 11%, and former Gov. Mitt Romney at 8%.

AFLTopDems190.png AFLTopReps190.png
Democrats Republicans

** Typos corrected.


Roundup: How Worthy Early Trial-Heat Polls?

Topics: 2008 , The 2008 Race

Here is a quick round-up of recent commentary and blogging elsewhere on polls and surveys on the issue of how much attention we should be paying to early trial-heat results:

  • In the latest update of his "Parse the Polls" feature, the Washington Post's Chris "The Fix" Cillizza asks three prominent partisan campaign pollsters whether "national surveys matter?" While the pollsters "all agreed that national polls have real meaning" for the 2008 primary races, Cillizza concludes;

[N]ational polls still lag behind early state polling when it comes to providing an up-to-the-minute look at the state of the race. It stands to reason that voters in Iowa who see the candidates on an almost daily basis -- either in person or on their television sets -- will have a deeper and better sense of who is truly viable than someone who lives in Connecticut or Idaho and occasionally sees a candidate on a news program.

  • Kathy Frankovic's latest CBS.com column provides some reasons "why we should be cautious - and maybe even a little skeptical - when reading poll measurements of candidate preference this early in a presidential campaign." One example:

Potential voters often choose candidates they are familiar with. Many announced candidates are simply unknown quantities. Even after his years in the Senate and a previous presidential run, 55 percent of Americans interviewed in an April Gallup poll still could not say whether they had a favorable or unfavorable opinion of Delaware Democratic Sen. Joe Biden. The earliest polls say more about name recognition than likely votes.

  • Last week in his Slate blog, Bruce Reed expressed even more cynicism about the usefulness of early primary polling. For all the poll bashing rhetoric, Reed did highlight one question I wish media pollsters would ask more often: Here is the full result from the most recent CNN/WMUR poll of New Hampshire Republicans (June 6-11, n=304):

"Have you definitely decided who you will vote for in the New Hampshire primary ... are you leaning toward someone ... or do you have no idea who you'll vote for?"

- 6% Definitely decided
- 37% Leaning toward someone
- 57% No idea who you'll vote for

Reed is right to point out the underlying discrepancy: Virtually all of New Hampshire's Republicans (92%) select a candidate when asked for whom they would vote if the election "were held today," yet a big chunk of these also say they have "no idea" who they will support. So it is not surprising to see results on the trial heat questions vary greatly from poll to poll and pollster to pollster.

Internal campaign pollsters always ask some variant of the "certainty" follow-up question, yet media pollsters rarely do. Reed urges them to "routinely report the more revealing percentage of voters who have no earthly idea whom they'll actually vote for." I couldn't agree more.


POLL: SurveyUSA Bloomberg Results


Additional results from recent SurveyUSA automated surveys featuring three-way presidential trial-heats including indepenant Mayor Michael Bloomberg. Each survey sampled approximately 500 registered voters per state. According to their analysis, a Bloomberg candidacy "siphons enough Republican votes to FLIP RED states Iowa, Missouri, Ohio, and New Mexico BLUE."


POLL: American Research Group Nevada Caucus


A new American Research Group statewide survey of likely voters in Nevada (conducted 6/15 through 6/19) finds:

  • Among 600 Democrats, Sen. Hillary Clinton (at 40%) leads Sen. Barack Obama and former Sen. John Edwards (both at 16%) in a statewide caucus.

  • Among 600 Republicans, former Gov. Mitt Romney edges out former Mayor Rudy Giuliani (23% to 21%) in a statewide caucus; Sen. John McCain and former Sen. Fred Thompson trail at 16% each.

See other Nevada Caucus poll data available on Pollster.com:

Democrats Republicans

Please note that we only plot trend lines when we have at least eight polls.


POLL: Quinnipiac New York State Primary/GE


A new Quinnipiac University statewide survey of registered voters in New York State (conducted 6/12 through 6/17) finds:

  • Among 447 Republicans, former Mayor Rudy Giuliani leads former Sen. Fred Thompson (46% to 14%) in a statewide primary; Sen. John McCain trails with 8%.

  • Among 546 Democrats, Sen. Hillary Clinton leads former V.P. Al Gore (44% to 18%) in a statewide primary; Sen. Barack Obama trails with 14%.

  • General election match-ups:

    Giuliani 37%, Clinton 52%
    Giuliani 43%, Obama 43%
    McCain 32%, Clinton 56%
    McCain 33%, Obama 48%


POLL: Mason-Dixon Iowa Caucus


A new Mason-Dixon statewide survey of likely voters in Iowa (conducted 6/13 through 6/16) finds:

  • Among 400 Democrats, Sen. Hillary Clinton edges out former Sen. John Edwards (22% to 21%) in an Iowa caucus; Sen. Barack Obama runs at 18%, Gov. Bill Richardson at 6%.

  • Among 400 Republicans, former Gov. Mitt Romney leads former Sen. Fred Thompson (25% to 17%) in an Iowa caucus; former Mayor Rudy Giuliani trails at 15%, former Gov. Mike Huckabee at 7%, Sen. Sam Brownback and Sen. John McCain both at 6%.

See Pollster.com's complete poll data and charts for the Democratic and Republican Iowa caucus.

AIATopDems195.png AIATopReps195.png
Democrats Republicans


New: Charts and Table for Early Presidential Primary States

Topics: Pollster.com

Today we launch pages on Pollster.com displaying charts and tables of public polling data (and links) for the Democratic and Republican contests in five early presidential primary states. For those who can't wait to dive right in, here are the links:

Each page has the same elements (and we will be rolling out equivalent versions for the national trial-heat data very soon). Here is a quick guided tour:

At the top of each page you will find a chart showing the "top contenders" (roughly 4-5 candidates in each state). You can see a larger size pop-up version by clicking on the chart. Clicking the "2007 data only" link just below the chart will bring up a large-size version showing more recent data.

Just below the chart is a table listing each of the individual polls charted in each state. The entry for each poll listed includes a link to the original source (if available), the survey dates, sample size and the percentages for the top contenders.

Since both races feature more candidates than we could fit into a legible table on our main page, we have also created similar tables in a suitable-for-printing PDF format that includes the data for all active and potential candidates included on each poll. Just click the link ("download printable PDF") below each table.

Finally, each page also includes a set of "small multiple" charts -- a graphic that features an identically formatted set of trend charts, with one chart for each candidate. Note that the number in parenthesis in the title of each small chart displays the current estimate of support for the candidates.

The trend lines are Charles Franklin's regression-based estimators, he explains those more detail here.

Finally, a special note: These charts are a work in progress. In many ways they serve as a prototype for what we are hoping to introduce over the next six months. So we would very much like to hear your feedback on what you like, what you don't, what needs improvement and what you would like to be able to see or do with the data that you cannot. So please email us with your thoughts, comments and suggestions.


About Our State Primary Trend Lines

Topics: 2008 , The 2008 Race

State level polling is vital for understanding the nomination contest. Candidates have to win in the states and of course the early states are especially vital. But the polling in states raises a variety of issues that should be kept in mind when reading these state primary charts.

First, the number of polls in a state varies widely and that affects our trend estimates. Ideally, the trend estimator should have a dozen or more polls to work with before we take the trend very seriously. When the number of polls drops too low, the trend estimator will jump around considerably if new polls are very far from previous polling and may produce jagged trend lines that are likely to change with more data. Despite this danger, we've estimated the trend with as few as eight polls, rather than stick to the safer minimum of twelve. Too many states are between 8 and 12 polls to ignore, and while we are cautious most of the trends look pretty reasonable even with less than 12 polls. When the polls are consistent with each other, the trend estimate will still be pretty good even with eight. But when there is substantial disagreement among the polls, we will get jagged or otherwise "bad" trends. Since the plots show the actual polls you can look at the data yourself and decide whether the trend is a reasonable fit to the data, or if it is erratic enough to be discounted. You decide. On the bright side, as more polls appear these trend estimates should all stabilize and this problem disappears.

The trend estimator here is the more stable and conservative estimator ("old blue"). This is because the generally small number of polls would make a less stable estimator jump around-- chasing random variation in the polls rather than giving a good estimate of the actual trend. The stable estimator may be a little slow to notice sharp changes in trend, though with the current (rather low) number of polls this will not be a big problem. When the number of polls grows enough, we'll check the more sensitive estimator as well.

Even with the stable estimator, a single poll at the end of the data can exert quite a bit of influence on the trend estimator. Be careful about extending the trends, especially if there is only a single recent poll after a period without polling. On the other hand, if the polls are consistent with each other then the trend estimator will follow them.

When there are fewer than eight polls, we don't try to compute a trend, and instead simply print the median of the existing polls. The median should be a pretty good estimate when the data are stable or scattered, but when there is a strong upward trend the median will tend to underestimate the candidate's current standing (and overestimate with a downward trend). Again, visual inspection of the data points should give you a good insight into this.

We are offering two "views" of the data. The "long-term" view consists of all data from January 1, 2005 through April 1 of 2008 (as those late data become available!) This gives the most comprehensive view of how the nomination races have shaped up and how they have changed over time. On the other hand, some people would like to "zoom-in" and focus just on recent data in 2007 and later. We give you that option, with a set of charts that run from January 1, 2007-April 1, 2008. You choose.

The trend-lines always use all the data regardless of which view you are looking at. Therefore the lines represent exactly the same numbers in either view. This can occasionally produce puzzles. A trend that fits the overall data in the long view may appear to not fit a small segment of data in the zoomed in view. A good example of this is Gingrich in Florida. In the long-term view, the trend line fits the data quite well but there are three polls at the start of 2007 that are well away from other polls and the trend line. When looking at the zoomed in view for Gingrich in Florida, the trend seems not to fit these data from early 2007, but it is actually the three polls that are "abnormal" not a poor fit of the trend line. When in doubt, check the long-term view.

If you switch between the long-term view (2005-2008) to the zoomed-in view (2007-08) be aware that the change in aspect ratio of the chart will mean the slopes of the trends will change. The numbers represented by the trend lines are exactly the same, but the slopes of the lines will be more shallow or less steep. Be careful judging a "sharp" change in the long-term which will look more gradual in the zoomed-in view. Only the aspect ratio has changed --- not the actual rate of change.

With state level polls there is often a great deal of variation across pollsters. Some polls may be excellent while others are poor, but it is hard to know which is which. Sample size can also vary considerably. This will produce more variation in polls than we might see with larger national samples of more consistent quality. This is why the trend estimator is probably more important in the state polls-- it will iron out the differences among polls. And given enough polls, the trend estimator will tend to ignore polls that are far from most of the other data. (However, it can be fooled if the bad poll is right at the end of the data, so be careful if the latest poll seems out of line with previous polls.)

We've written here before about the problems of sampling "caucus goers" or even likely primary voters. These problems are even worse than identifying likely voters in general elections. This should be expected to add more variability to the polls in the states. Again, the trend does its best to extract the signal from the noise.

Finally, different pollsters deal with the candidate list differently. Some exclude Gore while others include him. Some include Gingrich, others not. These are design decisions by the pollster, but may increase support for included candidates because the excluded candidate's supporters are forced to pick a candidate in the list. Our view of this is that the pollster has made their best judgment of how to ask the questions and we don't second guess that. But it means one pollster might have more support for top candidates than another because they have excluded, say, Gore who would otherwise get 10-15% support. If you want to know these details, go to the individual polls.


POLL: Rasmussen 6/19 Round-Up


New Rasmussen Reports automated surveys find:

  • Among 618 likely Republican primary voters, former Sen. Fred Thompson (at 28%) edges out former Mayor Rudy Giuliani (27%) in a national primary; Sen. John McCain and former Gov. Mitt Romney both trail at 10% (conducted 6/11 through 6/14).

  • Among 775 likely Democratic primary voters, Sen. Hillary Clinton leads Sen. Barack Obama (38% to 27%) in a national primary; former Sen. John Edwards trails at 16% (conducted 6/11 through 6/14).

  • Among 800 likely voters, Clinton leads Romney (50% to 41%) and edges out Thompson (48% to 43%) in nationwide general election match-ups.

  • Among 800 likely voters, Obama leads McCain (46% to 38%) in a nationwide general election match-up.


POLL: USA Today/Gallup Bush Approval

Topics: Pollsters

Additional analysis to the recent USAToday/Gallup national survey of 1,007 adults (conducted 6/11 through 6/14) finds 32% of Americans approve of the way George Bush is handling his job as president -- "the most negative of his entire presidency and is just one point higher than his all-time low as president;" 65% dissaprove.


On Dog Whistle Effects and Immigration Polling


Last Friday, Mickey Kaus noticed a "stark conflict" between the recent NBC/Wall Street Journal, Los Angeles Times and CBS/New York Times polls on immigration that "demands a Mystery Pollster explanation." Lets see what we can do.

The questions concern what Kaus calls the "legalization plank" of the immigration legislation before Congress. The challenge in interpreting any of these results is to remember that most Americans have, at best, only a vague sense of what the immigration bill does, much less which provisions they favor. As I wrote previously (especially here and here), survey questions about specific proposals largely measure reactions, not preexisting opinions about the proposals. Respondents tend to listen to the text of the question and form opinions on the spot. As such, the results can vary greatly depending on the way the pollster asks the question.

One particular contrast illustrates how minor differences in wording can produce dramatic differences in the results: Consider these two questions

NBC/WSJ (June 8-11, n=1,008 adults) - Allowing illegal workers who arrived in the U.S. before January first of this year to receive an automatic work visa if they pay a fine of around five thousand dollars.

- 30% Strongly/somewhat favor
- 64% Strongly/somewhat oppose
- 6% Not sure

CBS/NYT (May 18-23, n=1,125 adults) - Would you favor or oppose allowing illegal immigrants who came into the country before January to apply for a four-year visa that could be renewed, as long as they pay a $5,000 fine, a fee, show a clean work record and pass a criminal background check.

- 67% Favor
- 28% Oppose
- 5% DK/NA

Both ask about essentially the same provision in the bill, and include many of the same elements of that proposal, yet show diametrically opposite results. Look more closely, however, and the questions differ in potentially crucial ways:

  • The NBC/WSJ question describes the "work visa" as "automatic," while the CBS/NYT question says it is "a four year visa that could be renewed." I have no idea whether the every-four-year renewal procedure is "automatic" or not, but I would wager that few CBS/NYT respondents heard it that way.

  • The CBS/NYT question describes requirements for a "clean work record" and "criminal background check," elements not mentioned in the NBC/WSJ question.

  • Do respondents assume an "illegal worker" is the same thing as an "illegal immigrant?" Presumably, "illegal" is the key word in both, but perhaps the two phrases conjure different images.

Unfortunately, media and campaign pollsters know little about how respondents hear these sorts of questions or what pieces of information they weigh most heavily in their answers. We often refer to these sorts of differences as "dog whistle effects" -- the respondents seem to hear things we miss. There is a way to "debug" this sort of question (hint: Google "cognitive pretesting"), but it requires far too much lead time and costs far, far too much to be practical for media and campaign pollsters. So we are left to speculate about how to interpret the results.

However, let me suggest this rule of thumb for interpreting results of different survey questions that ask about essentially the same proposal: The more consistent the results, the more likely that we are measuring true, pre-existing opinions about the proposal itself. The more results tend to diverge, the greater the odds that respondents are confronting the proposal for the first time and are simply reacting, drawing upon real (and sometimes conflicting) attitudes triggered by the information provided in each question.

What's the difference? I suspect that in this case, we will hear from partisans on both sides of the immigration debate as to which language is best and which "flawed." But in that case, we are no longer debating public opinion, but rather the most accurate way to characterize the bill. So in this case, the debate about the polls is mostly a debate about the bill itself.


POLL: SurveyUSA 16 Giuliani vs. Clinton Match-Ups


New SurveyUSA automated surveys in 16 testing two-way general election match-ups pitting former Mayor Rudy Giuliani against Sen. Hillary Clinton find:

StateGiulianiClinton
Alabama5341
California4449
Iowa4147
Kansas5341
Kentucky4744
Massachusetts4252
Minnesota4150
Missouri48 4747 46
New Mexico4450
New York3856
Ohio49 4645 49
Oregon48 4446 48
Texas5437
Virginia4844
Washington48 4744
Wisconsin49 4643 47

SurveyUSA's three-way match-ups between Giuliani, Clinton, and Mayor Michael Bloomberg are available here.

** Typos corrected.


POLL: Cook/RT 2008 Primary


Thanks to Charlie Cook, who shares with Pollster.com readers this exclusive sneak-peak at results from the latest Cook Political/RT Strategies national omnibus poll of registered voters (conducted 6/15 through 6/17):

  • Among approximately 380 Democrats and Democratic leaners, Sen. Hillary Clinton leads Sen. Barack Obama (30% to 20%) in a national primary; former Sen. John Edwards trails at 13%, former V.P. Al Gore at 9%.

  • When Gore is excluded, Clinton leads Obama (32% to 22%), while Edwards trails at 16%.

  • Among approximately 330 Republicans and Republican leaners, former Mayor Rudy Giuliani edges out Sen. John McCain (20% to 16%) in a national primary; former Sen. Fred Thompson trails at 15%.


POLL: SurveyUSA 16 3-Way General Election Match-Ups


New SurveyUSA automated surveys in 16 states testing three-way general election match-ups between former Mayor Rudy Giuliani, Sen. Hillary Clinton, and Mayor Michael Bloomberg find:

StateGiulianiClintonBloomberg
Alabama463911
California404510
Iowa374211
Kansas47368
Kentucky424110
Massachusetts37479
Minnesota37487
Missouri394410
New Mexico41458
New York324915
Ohio41478
Oregon384411
Texas483410
Virginia45409
Washington414211
Wisconsin404410


POLL: USA Today/Gallup '08 Primary


A new USA Today/Gallup national survey (story, analysis) of 1,007 adults (conducted 6/11 through 6/14) finds:

  • Among 516 Democrats and Democratic leaners, Sen. Hillary Clinton "has regained a double-digit lead [33% to 21%] over Illinois Sen. Barack Obama in the USA TODAY/Gallup Poll two weeks after the survey found the Democratic presidential rivals essentially tied" in a national primary; former V.P. Al Gore trails with 18%, former Sen. John Edwards with 11%, and Gov. Bill Richardson with 5%.

  • Among 393 Republicans and Republican leaners, former Mayor Rudy Giuliani leads former Sen. Fred Thompson (28% to 19%) in a national primary; Sen. John McCain trails with 18%, former Gov. Mitt Romney and former Speaker Newt Gingrich both with 7%.


POLL: Mason-Dixon South Carolina


A new Mason-Dixon statewide survey of likely primary voters in South Carolina (conducted 6/13 through 6/15) finds:

  • Among 329 Democrats, Sen. Barack Obama leads Sen. Hillary Clinton 34% to 25% in a statewide primary; former Sen. John Edwards trails with 12%.

  • Among 432 Republicans, former Sen. Fred Thompson edges out former Mayor Rudy Giuliani 25% to 21% in a statewide primary; former Gov. Mitt Romney trails at 11%, Sen. John McCain at 7%, former Gov. Mike Huckabee at 5%.


 

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR