Mark Blumenthal | September 15, 2006
More details coming soon!
More details coming soon!
[Editor's note: This post is the first on Pollster.com from our partner Professor Charles Franklin. Charles also blogs at Political Arithmetik, but will also be a regular contributor here. We will be changing the menus and titles next week to designate his contributions approrpriately, but the authors name always appears at the bottom of each post].
Approval of President Bush's handling of his job has taken a significant turn up since mid-August. In ten polls completed since September 1, approval has averaged 39.9%, while disapproval has averaged 55.7%.
Approval around 40% is not particularly good news for any President, but this represents a considerable improvement since earlier in the year.
The chart here plots each of 336 national polls taken since January 1, 2005 that asked a presidential job approval question. The dots represent the poll's approval rate. There is a good deal of scatter in these points, representing the variation due to sampling error, questionnaire effects, survey organization practices, interviewer variability and everything else that causes surveys to vary. While the individual polls vary quite a bit, the trend in approval over the last 21 months is clear. The blue line estimates the trend in approval, allowing each poll to have its say, but keeping to the middle of the range of polls at any point in time, so that the average deviation around the trend line is zero-- the polls above cancel the polls below the trend.
After starting the second term at 50.5% approval, the President's standing with the public declined steadily until November 2005, when it rallied briefly before again sinking to an all time low in May 2006. Following the President's television address on immigration on May 15, however, approval again rallied. The summer produced a steady plateau of approval, but starting around August 15, the trend has once more moved up. The total movement since May has erased most of the ground lost since January 2006, though approval still remains some 10 points below where it stood at the beginning of the second term.
As of polling completed September 14, the trend estimate of approval stands at 40.3%, slightly above the average of the most recent 10 polls. Most of the summer was spent around 37%, so the current standing is a gain of a little over 3 points in a month. The all time low came on May 12 when the approval trend hit 33.98%.
Notice that around that time, polls ranged from a low of 29% to a high of 38%, a good example of how much variation there is across surveys. If one wished to make the President's improvement seem large, one could compare the all time low of 29% with the current highest reading of 43% and conclude there has been a 14 point run-up in approval. If, conversely one wished to minimize the improvement, one might compare the high in mid-may of 38% with the lowest recent reading of 37% and conclude that in fact there has been a one point decline in approval. Both of those comparisons are misleading, of course. The virtue of focusing on the trend estimate is that it is not so sensitive to any single poll, nor to a choice of which poll to compare to which. Based on the trend, the President's approval has risen 6.3% since May 12. Or put in a different way, approval fell by 16.5 points from January 2005 through May 12, 2006. Since then, over six of those points have been regained. This leaves the President 10.2 points below where he started the second term, but with a strong upward trend over the past 4 months.
The interesting question is whether this trend will continue, accelerate, flatten or turn down over the next 53 days until the election, and how much the President's approval rating will affect the vote.
Our Election Scorecard update on Slate for today moves the Virginia Senate race from solid to lean Republican after a new SurveyUSA poll further narrows George Allen's lead on the five-poll average.
One more tribute to the late Warren Mitofsky: The New York Chapter of the American Association for Public Opinion Research (NYAAPOR) has posted the remarks of several speakers at his September 5 memorial service. These include tributes from friends and colleagues who knew him best, including Kathy Frankovic and Marty Plissner of CBS News and Andy Kohut of the Pew Research Center.
Now that the Maryland primary is over, we have posted available polling data for the Senate race between Rep. Ben Cardin (D) and Lt. Gov. Michael Steele (R). The only remaining statewide general election contest where we are not yet posting available polling data to our charts section is the contest for governor in Massachusetts. However, we have included a link to a just released KRC/Communications Research poll (that tests all three potential Democratic nominees) in the "Recent Polls" box. The Massachusetts primary will be held next Tuesday.
A quick update on last night's Republican Senate primary in Rhode Island. Incumbent Senator Lincoln Chafee defeated challenger Stephen Laffey by an eight percentage point margin (53.6% to 46.4%). So in the battle of the polls -- described in my post on Monday -- Republican polling firm Public Opinion Strategies (POS) came far closer to the final result than the poll conducted by the Bureau of Government Research at Rhode Island College (RIC). The POS poll conducted two weeks ago had Chafee ahead by fourteen points (53% to 39%). The RIC poll done at exactly the same time had Chafee seventeen points down (34% to 51%). Given the wide difference, it seems obvious that polls for this particular race were sensitive to the way the pollsters defined likely voters.
Since Public Opinion Strategies has given away little about the methods they used to conduct their survey, this episode can teach us little beyond underscoring how challenging political polling can be in primary elections.
Our daily Senate Race Summary is now up on the Election Scorecard at Slate and focuses on two news polls in Arizona & Washington State.
Primary elections are underway today in nine states (AZ, DE, MD, MN, NH, NY, RI, VT & WI) plus the District of Columbia. ABC's The Note reports that "there are no network exit polls" being conducted in any of these states, but their poll closing information links to official sites that will provide election returns and a run-down on each of the key races.
Regular readers will remember a similar pronouncement from The Note regarding last month's Connecticut primary. "No pooled exit polls," they told us, only to be surprised along with everyone else when CBS News released results from its own experimental exit poll conducted outside of the official exit pool consortium.
So we won't say never. But for now, no signs of exit polls anywhere.
Another note on the Zogby Interactive Surveys. Crosstabs.org GOP pollster Rob Autry (whose company, Public Opinion Strategies, polls for Joe Lieberman) notes that Zogby's Connecticut survey asked a two-way vote question pitting Democrat Ned Lamont against Republican Alan Schlesinger before asking the five-way vote question featuring "Independent Joe Lieberman, Green Party Ralph Ferrucci, and Independent Party John Mertens."
Autry is right to assume that question order likely depresses Lieberman's vote in this instance. Checking the subscription-only Zogby crosstabs, he notes that Lamont receives 90% of the vote from Democrats on the first (two-way) question and then retains 77% of Democrats on the second (five-way) question. Compare that to the August survey from Quinnipiac, where Lamont received only 63% of the vote from Democrats on a three-way ballot against Lieberman and Schlesinger (Q11).
Of course, if we are going to scrutinize question order, we might also take a closer look at that Quinnipiac survey that waited to ask the Senate question until after favorable ratings that describe Lamont as a "businessman," Schlesinger as a "former state representative," and Lieberman as a "United States Senator." Unless those labels appear on the ballot, that particular question order is also a bit problematic.
Our daily Slate Scorecard update posted earlier this evening focuses on the new poll from Mason-Dixon that shows a narrowing race in the Virginia Senate pitting incumbent Republican George Allen against Democratic challenger Jim Webb.
We also discuss why the Slate Scorecard does not include the online Zogby Interactive/Wall Street Journal polls and made the following observation:
The latest Zogby results for Virginia-showing Webb ahead 50 percent to 43 percent-help explain our caution. Zogby's Virginia samples have been consistently more favorable to Webb than other pollsters, suggesting a bias in Zogby's online methodology.
With the help of Charles Franklin, here is a chart showing the consistent difference in Virginia. It plots the Allen margin (that is, Allen's percentage of the vote minus Webb's percentage -- click on the graph for a full size image) for each of the four pollsters that have tracked the race. All four show the same sharp drop in Allen's lead since July, but the Zogby result (the green line) has been consistently more favorable to Webb than the three telephone pollsters that use random probability samples of all telephone households rather than samples of Internet volunteers.
The differences are not trivial. The latest polls from Survey-USA, Mason-Dixon and Rasmussen have shown Allen with leads of 3, 4 and 5 percentage points respectively. Zogby's result is very different, showing Webb with a seven percentage point lead.
Incidentally, we do include all of the Zogby Interactive results and other Internet polls in the charts and tables here on Pollster.com. Our aim is to give readers the ability to compare results across pollsters. One minor wrinkle -- at least for now -- is that the 5-poll averages reported here may differ from those on Slate because the averages here include the Zogby numbers while those reported on Slate do not. We are hoping to address that conflict in a future update to our chart pages.
My last post involved what appears to be an example of true push polling in tomorrow's Republican Senate primary in Rhode Island. Although I nearly missed it in the chaos of Pollster's debut, two true surveys in that contest also provide a classic story of oddly conflicting poll results. In the contest between incumbent Lincoln Chafee and his challenger Steve Laffey, one poll had Laffey ahead by seventeen percentage points, another conducted at exactly the same time Chafee ahead by fourteen. How can that be? The most probable explanation is the difference in the way each pollster selected their sample of "likely voters."
The first survey was conducted by the Bureau of Government Research and Services at Rhode Island College (RIC) from August 28 to August 30 (release & results). It showed Chafee trailing Laffey, 34% to 51%. The survey sampled 363 likely primary voters and reported margin of sampling error of 5%.
"A few hours later," according to the Political Wire of the Wall Street Journal Online, "the National Republican Senatorial Committee [NRSC], which supports Mr. Chafee, released its own internal poll done by Public Opinion Strategies [POS] showing the incumbent leading 53% to 39%." The POS survey was fielded over exactly the same evenings (August 28-30). It had a sample size of 400 likely voters and a margin of sampling error of 5%.
As should be obvious, random sampling error alone does not explain the difference. The conflicting results, combined with the high profile push on Chafee's behalf by the NRSC, has raised a few eyebrows. While conspiracy theories always seem to find a receptive audience in the Internet, I am dubious, here's why:
The methodology of the Rhode Island College poll is explained in their release, which explains that they drew their sample from "the most recent updated voting lists provided by the Office of the Secretary of State," limiting selections to registered Republicans and unaffiliated voters. According to their questionnaire, they asked respondents if they were "likely to vote in the Democratic Primary, the Republican Primary, or will you wait to vote in the November General Election?" Those who said they would vote in the Republican primary were then defined as likely voters and interviewed.
This one question approach to screening for likely voters is a bit unusual, at least in my experience. Most pre-election polls - especially those in low turnout primaries - typically take a few extra steps to try to narrow the respondent pool given the well documented tendency of respondents to over-report their intent to vote. These steps might include using questions about past vote experience or interest in the campaign to further screen out likely non-voters voters or perhaps using the actual past vote history on the voter file to narrow the sample.
I spoke briefly to Dr. Victor Profughi, the director of the Rhode Island College poll, about their likely voter model. He described it as the "same process we have been using to screen primary voters for a number of years." He also estimated that roughly two thirds of contacted respondents were screened out by their question about primary voting.
If the NRSC or Public Opinion Strategies has released information about their sampling methodology, I have been unable to find it. However, the WSJ Washington Wire item included an "intriguing nugget" that provides a pretty good clue about the discrepancy between the surveys: "Of the 53% of respondents who could actually name the primary election date, 58% support Mr. Chafee compared to 37% who back Mr. Laffey."
So Chafee's lead was wider on the POS poll among those who knew the primary date (+21 points) than among those who did not (+6 points by my calculation). True likely voters tend to be more knowledgeable about elections (one reason the Gallup likely voter model includes similar measures of knowledge). This result suggests that Chafee does better among the most likely of likely voters, a difference that may help explain the gap between the two polls.
Primary elections often present pollsters with a huge challenge because the proportion of eligible voters that turn out is typically very low. The Providence Journal reported over the weekend that "the previous modern GOP record primary turnout came in 1994, when 45,023 voters participated in the gubernatorial primary." That turnout was roughly 9% of the pool of registered voters in 1994. A comparable turnout tomorrow would amount to roughly 13% of the eligible Republicans and unaffiliated registrants.
What this tells us about tomorrow's results is anyone's guess, but I'm sure a lot of pollsters will be watching closely.