March 30, 2008 - April 5, 2008
Mark Penn meets with the Columbia's U.S. ambassador to discuss a free trade agreement then admits to an "error in judgement," drawing comment across the blogosphere.
Annette John-Hall (of the Philadelphia Inquirer) considers "those crazy polls" in Pennsylvania.
David Leonhardt sees evidence in the NY Times/CBS poll that Obama supporters are more optimistic about the future than Clinton supporters.
Frank Newport notes that Republicans oppose a federal government bailout to "to help prevent people from losing their homes because they can't pay their mortgage."
David Sirota plots Barack Obama's support in the primary states against the rank of each state's black population and sees evidence that "racism is a powerful force" in the Democratic contest.
[And Bob Somerby points out that the middle states in Sirota's chart held caucuses, not primaries (via comment by Pollster reader kingsbridge77)].
Brendan Nyhan finds problems with Sirota's graphs and puts forward nine more of his own (both items via TPM).
Michael McDonald (PDF) finds that a mail-only revote in Florida would "cause disproportionate problems with the delivery of mail ballots to African-Americans" (via John Sides).
PPP gets a call from "Barack Obama."
My colleague Matthew Gottlieb, polling editor of The Hotline, has posted a more in-depth analysis of the Diageo-Hotline poll (data, release) that we linked to earlier today. Some highlights:
Meanwhile in a McCain-Obama matchup, McCain leads 46%-44% – within the margin of error but a shift from Obama’s 48-40% lead in Feb. Since the previous poll, McCain’s support among Dems has nearly doubled from 8% to 15%; at the same time, Obama’s lead among Dems has dropped from 76% to 53%. McCain also bests Obama by 12% among Inds, although no trend data is available for this group.
Interestingly, in a general-election matchup with McCain, Obama performs better than Clinton among women – he leads McCain by 5%, while McCain and Clinton break even, 45%-45%. Additionally, in a McCain-Obama matchup, Obama carries 94% of black votes, but in a McCain-Clinton matchup, just 74% vote Clinton. What happens to this 20% of blacks? At least 9% defect to McCain, and 6% remain undecided. Another 6% refuse to answer – perhaps they plan to stay home in the event Obama is not on the ballot in Nov.
Speaking of these “revenge voters” – the potential calamities of the divisive Dem primary – the Diageo/Hotline poll offers further evidence they exist. In the McCain-Clinton Nov. matchup, 19% of Obama primary voters opt for McCain, versus 14% of Dems overall. The damage is even more pronounced in a McCain-Obama matchup, 29% of Clinton voters select McCain, compared to 15% of Dems overall.
He also notes a drop in the favorable ratings of both Clintons:
One of the most remarkable statistics in the latest poll is the drop in favorability suffered by both Hillary and Bill Clinton. Both the Senator and ex-POTUS now have net unfavorable ratings among RVs. He moved from a fav/unfav of 55%/41% in Feb. to 45%/51% now, while Hillary dropped from 54%/40% to 43%/53% over the same period.
I checked the favorable ratings among the 44% of registered voters that self-identify as Democratic primary voters (say they "typically vote in primary elections for national, state and local office [for] the Democratic Party candidates" - Q16).
- Since January, Hillary Clinton's favorable rating among Democratic primary voters dropped from 88% to 70%, while her unfavorable rating increased from 12% to 26%. From January to March, her "strongly favorable" rating declined from 50% to 30%.
- Over the same period, Barack Obama's ratings among Democratic primary voters remained roughly constant: 76% favorable, 18% unfavorable in January, 77% favorable, 17% unfavorable now. His very favorable rating has also changed little from January (42%) though March (44%).
See Gottlieb's Hotline OnCall analysis for much more.
Greenberg Quinlan Rosner (D)
Generic Pres Ballot
Dem 53, Rep 41
Generic House Ballot
Dem 53, Rep 40
Dem Incumbent: Dem 69, Rep 26
Rep Incumbent: Rep 53, Dem 38
Full results here.
Diageo / Hotline
National end date 3/31 (2/17)
Obama 50, Clinton 38 (was Clinton 45, Obama 43)
McCain 46, Obama 44 (was Obama 48, McCain 40)
McCain 50, Clinton 41 (was McCain 48, Clinton 40)
79% of adults have a land line connected to their residence (81% in 2006).
14% of adults only use a cell phone at their residence (11% in 2006).
Full analysis here.
American Research Group
Clinton 53, Obama 44
CBS News/New York Times
(CBS story, economy, campaign, race relations;
Times story, results)
Obama 46, Clinton 43
Obama 47, McCain 42... Clinton 48, McCain 43
Clinton 51, Obama 41
Clinton 49, Obama 46
I am posting the data in the following table partly because Doug Schwartz, director of the Quinnipiac University Poll, was kind enough to share it, and partly because it tends to confirm a point I made two weeks ago about the state of play in Pennsylvania's Democratic primary race. I wrote that Barack Obama had dropped among college educated white voters there (in a survey conducted just before and after the Jeremiah Wright controversy emerged, but before Obama's speech) and had room to rebound:
How [Obama] ultimately fares among Pennsylvania's college-educated Democrats could well determine whether he loses Pennsylvania narrowly (as his campaign forecast in early February) or by a double-digit margin. If, hypothetically, Obama wins his usual overwhelming majority among Pennsylvania's black Democrats, and Clinton racks up the same 40-point margin among non-college whites that she did in Ohio, Obama can still run within10 points overall if he can best Clinton by at least 4 points among college-educated white voters.
As it turns out, that seems to be what has happened, at least on the latest Quinnipiac poll, which shows the biggest net shift occurring among college educated white voters. Although the change appears to be just shy of statistical significance, Obama now holds a 5-point advantage (49% to 44%) over Hillary Clinton among college educated white voters, an improvement since March but roughly the same margin on their two February surveys. That shift was just large enough to reduce Clinton's overall lead to nine points.
Meanwhile, the preferences of non-college white voters have shown little or no change since February in the Quinnipiac surveys, a finding that illustrates the strength of Clinton's position in Pennsylvania and the difficulty Obama will have in further narrowing her lead.
Of course, the Quinnipiac survey was fielded over eight days, mostly last week (March 24-31). As Pollster readers know, three new automated surveys have fielded this week (fielding for one or two days each) that show a closer race. A fourth poll, conducted by SurveyUSA this past weekend, showed Clinton leading by 12 points. None of these surveys provide crosstabulations by education, but all (that have polled previously in Pennsylvania) show some narrowing of the race.
Clinton 45, Obama 43
Note, this document says 45.3 to 42.4 and this document says 45 to 43.
Fairleigh Dickinson University's Public Mind Poll
Obama 47, McCain 42... Clinton 48, McCain 43
My NationalJournal.com column , which discusses the CBS panel survey conducted after the Barack Obama's speech on race and Jeremiah Wright and the attack on it by Matt Towery of InsiderAdvantage, is now online.
If this topic is of interest, you will also want to read Kathy Frankovic's column from last week, which discussed more of the philosophy behind panel-back surveys and includes specific findings from the follow-up to the Obama speech.
Gary Langer averages airworthy national polls and considers coverage problems in a "global" poll.
Kathy Frankovic pays homage to Phil Meyer's contribution and ponders the future of polls and journalism.
Frank Newport sees little change in patterns of voting by religion.
David Hill thinks John McCain wants to run against Barack Obama.
Mark Mellman warns against March polls to forecast November elections.
Anthony Greenwald expounds on "Bradley" and "Reverse Bradley" effects.
Noam Scheiber links to Greenwald and takes some due credit.
Seelye and Bosman provide some historical context on Democratic defectors (via Schaffner).
The Coalition for a Democratic Workplace/
McLaughlin & Associates (R)
Udall, Schaffer 32
Collins 54, Allen 31
Coleman 46, Franken 40
New analysis from Gallup looks at data collected among 19,076 registered voters between 3/7 through 3/29 and finds:
"Obama's strength is his appeal to black voters, and his somewhat greater appeal than Clinton's to independents and Republicans. On the other hand, although Clinton attracts the support of a lower percentage of blacks than Obama, she has a stronger appeal to white Democrats, particularly white conservative Democrats, only half of whom at this point say they would vote for Obama if he were the nominee pitted against McCain."
Read the full analysis here.
Public Policy Polling (D)
Obama 45, Clinton 43
Clinton 50, Obama 41
Obama 43, McCain 39... Clinton 48, McCain 40
McCain 46, Obama 37... Clinton 44, McCain 42
Obama 43, McCain 42... Clinton 48, McCain 39
Yesterday, in a burst of blogger exuberance, I posted some charts emailed by my long ago employer Harrison Hickman, the Democratic pollster now associated with the firm Global Strategy Group who also conducted surveys earlier this year for John Edwards. I gave Hickman and his associate credit for the charts but then provided my own interpretation, comments I subsequently qualified. This morning, I did what I should have done in the first place, which is offer Hickman the opportunity to describe the charts in his own words. Harrison's summary follows below.
The initial (and only) purpose of the line charts Ben Margolis and I sent Mark yesterday seems to have been obscured by our failure to provide explanation with the charts and some of the verbal vines their publication stimulated ("Day-of-Week Effect in Gallup Daily?"). Hopefully I can provide something of a corrective for the former.
1. We submitted the charts without explanation but with obvious doubts about their significance, statistical and otherwise. The subject line of my original e-mail to Mark was "spurious or what?"
2. The point of the exercise was to note that all the hoopla about Obama or Clinton being ahead or behind by more or less at a specific point was ignoring a persistent pattern in the data. (The time period covered was since the departure of the Sainted Senator Edwards.) The point was that the hoopla was misguided, not that the pattern itself is all-telling. We certainly never intended to suggest that particular changes could be associated with specific days or that there was any iron law of anything at work. Our message: if you don't like the results today, wait a couple of days. If you do, it might be wise to exercise some restraint. In that vein, Mark is correct in urging caution about reading too much into day-to-day changes. I would urge similar caution in the interpretation of two techniques under discussion here.
3. The rolling average technique was developed to introduce a cost-effective way to report opinion data more or less continually in critical points of a campaign, and there are a variety of different ways to calculate those averages. But it is important to note that the "smoothing" artifact is the reason the technique is useful, not the reason it is misleading. An on-going series of one-day polls would be more misleading for campaign professionals and poll consumers than rolling averages.
4. Perhaps the most important statistical point to understand about these types of polls is that a sample is not a sample until it is completed. Before its completion, a sample is not "random" even in the colloquial sense of the term. It is for this reason that no one should mistake the partial results of stand-alone samples as precise, no matter how extensively those partial data are weighted. This is particularly important to remember when confronted with early wave results of election day polls (exit polls).
5. One should be mindful of but not obsessed with any particular statistical test. Estimation error is the most reported but hardly the only type of error in opinion research. It is treated as more important than it is and than the other types of errors because (a) it has the veneer of precision because it is a number and (b) it easier to understand and better researched than other categories of errors. Here is a simple measure of the its importance: "sampling error" so-called is taught in the introductory course but other types of errors are saved for later in a student's learning. Here's another: If you read any questionnaire carefully and think seriously about the methods used to gather the data, you almost always will find sources for potentially greater "error" than estimation error in what is reported.
6. In fact, a legitimate argument can be made that estimation errors are not really an applicable statistic for most opinion polls we see. The underlying assumption of sampling error is that the sample in question is random, and random has a very precise statistical definition. For a host of reasons, the samples in most polls do not qualify as random in a strict sense and, in too many cases, even under the loosest standards. Harris or Gallup (forgive me for mot remembering which) used to report a table of mathematical estimation error ranges but also something called ranges generated "from observation." I do not recall that they ever explained the source of the observations but found the presentation refreshing as an implicit statement about the limitations of statistical error calculations.
7. Two final observations from reading comments. As consumers and practitioners, recognize that political arguments are still political arguments even when they are dressed up with statistical language. And, finally, do not assume that any pollster is part of a larger conspiracy against your preferred candidate until you have ruled out (a) incompetence and (b) the possibility that things are not as rosy as you want them to be.
Global Strategy Group, LLC
P.S. Not to suggest that there is a day-(or period-)of-the-week effect in the Gallup data, but as of a few minutes ago, it seems that the stop-the-presses 10-point "lead" Obama enjoyed this weekend is now four points. An up-to-date version of our original charts is below, including a line based on calculation beginning the week after Super Tuesday.
Gallup's Jeff Jones responds to my post yesterday suggesting a possible day-of-week effect in the Gallup Daily tracking of the Obama-Clinton race:
We appreciate the interest people have in Gallup Daily tracking data. Your hypothesis of possible day-of-week effects in Gallup’s tracking poll is based on suggestive evidence from Gallup’s published three-day rolling averages. However, looking precisely at the results for each individual day of the week, we find no such day-of-week effect.
The table below shows the results by day of week, consisting of over 36,000 interviews with Democratic voters conducted from Jan. 2 through March 31.
Over the course of tracking, the average levels of support are 45% Clinton, 42% Obama. For any given day of the week, the average is within one point of the overall average for each candidate.
One poster to your site suggested a day-of-the week effect may have been a more recent phenomenon. But even when you look at the data on a monthly basis, where each day of the week occurs only 4-5 times, the same general pattern appears – the average for a candidate on any specific day of the week is within 2 percentage points of the average for the entire month. There is one exception to that, Clinton has had some good Mondays in March, basically every other one, but even with that her Monday average is only 3 points higher than her overall March average.
The possibility of a day of the week effect has come up in relation to prior Gallup tracking data, such as for the 1996 and 2000 elections. We carefully examined those data for evidence of such an effect, and did not find anything to suggest a systematic effect.
A quick note to regular readers that I updated my original post on the Gallup Daily tracking below with some second thoughts about the statistical significance of the apparent day-of-week effect. While some pattern may exist, the evidence available is not strong enough rule the role of random chance as an explanation. See "Update 2" in the original post for the details.
One point that is also worth clarifying: Others have read my observations on this and related subjects as evidence of a "flaw" in the Gallup Daily survey or as an argument that we ignore the Gallup Daily data altogether. I would not go that far, though I have been consistent in recommending (here, here and here) that we make less out of the day-to-day variation than we typically do. Any day-of-week effect, for example, would disappear if we compared weekly averages rather than looking for day-to-day change.
Update: Gallup's Jeff Jones responds with data since January broken out by day-of-week.
Clinton 53, Obama 41
I received some interesting charts this afternoon from Harrison Hickman and Ben Margolis, both of the Democratic firm Global Strategy Group that polled for John Edwards until he withdrew from the race (full disclosure: Hickman was my employer longer ago than either of us wants to admit). When Hickman and Margolis plotted the Gallup Daily results for the Obama-Clinton race, they noticed an impressively consistent pattern by day of the week. In data released during February and March, Obama typically does best on three-day samples that end on Saturday (combining interviews from Thursday, Friday and Saturday), while Clinton typically does best on samples ending on Tuesday or Wednesday night (covering Sunday through Wednesday).
The chart below shows the pattern in the Clinton lead (Clinton minus Obama) for each of the last nine weeks:
Hickman and Margolis sent a second chart which shows the average for each daily release across the nine-week period. On average, the results have shown a more than three point shift from Sunday-Monday-Tuesday (Clinton ahead by 1.3) to Thursday-Friday-Saturday (Obama ahead by 2.6).
So, given this pattern, do not be surprised if the ten-point Obama lead reported yesterday (based on a interviews from Thursday through Saturday) narrows a bit more by mid-week from the eight -point lead reported today. The results also suggest that we would get a less "volatile" sense of the race by looking at a seven-day rolling average that eliminates the apparent day-of-week effect.
The more interesting question is, why are we seeing this pattern? It seems counterintuitive, at least at first blush. I would have expected Obama's supporters to be harder to reach on weekends, given that they tend to be younger than Clinton supporters. Back in December, Obama's supporters seemed to be harder to reach around Thanksgiving and the Christmas holiday. Unless I am missing something obvious, this pattern is different.
Any thoughts? Theories?
UPDATE: Hold the, er, phone. I sent Hickman and Margolis a spreadsheet with the Rasmussen Reports daily tracking and they generated comparable charts. Oddly enough, the pattern is almost the mirror opposite of Gallup. On the Rasmussen automated tracking (which reports a four-day rolling averge), Obama does consistently better in the middle of the week, Clinton better on the weekends. Here is the chart of each week:
And here are the average values for each daily release:
I will have a chance to add more thoughts later, but I think the bottom line is that the conflict of the patterns shown by the two pollsters suggests that the apparent day-of-week effects are about differences in poll methodology, not a real variation in voter preferences over the course of the week.
UPDATE 2: Both in comments and in email, readers have questioned the statistical significance of the patterns in the charts above. I am persuaded that they have a point.
One argument many are making, with good cause, is that the values for adjacent days are not statistically independent. In other words, the value plotted for each day for Gallup represents a rolling average of three days of data (and for Rasmussen, four days). So it is not surprising to see smooth day-to-day trends. That's the point of the rolling average, something I've observed previously.
We can, however, isolate two independent measurements each week for Gallup from the available data. For example, there is no overlap between the data collected on Sunday, Monday and Tuesday (labeled "Tuesday" on the chart above) and on Thursday, Friday and Saturday (labeled "Saturday). Looking at the first chart above, we see what certainly looks like a "consistent" pattern: In seven out of eight weeks, Clinton does at least a point better in the Sunday-Monday-Tuesday sample than than the Thursday-Friday-Saturday sample.
If random chance were the only factor at work, then the odds of seeing a difference one way or another between the two independent samples in any given week should be like flipping a coin. The odds of flipping a coin and having it come up the same way seven times (either heads or tails) is about 7% -- highly improbable but still slightly higher than 5%, the level of confidence we usually require to call something "statistically significant."
However, as my friend Mark Lindeman tells me, my test is sketchy in several ways. Tuesday was primary day in many of these weeks. The biggest drop on the chart, for example, occurs during the first week of February following Super Tuesday. If we throw out just that one week, then odds of the late week drop occurring by chance alone on the Gallup series increases to about 13%.
Moreover, if we look at all eight weeks, but compare the Clinton margin each Sunday-Monday-Tuesday sample to the preceding Thursday-Friday-Saturday, the consistency seen in the chart disappears: Four ups, two downs and two comparisons that show no change. The odds of getting that pattern by chance alone are not much better than 50-50.
Things are a bit more complicated with the Rasmussen data, since the four-day rolling average makes it impossible for us to isolate two independent measurements each week, but the pattern illustrated in the chart is less consistent by week than for Gallup.
Others may suggest better approaches to the statistics and, of course, patterns in future weeks will add further evidence one way or the other. However, I am convinced that these patterns owe more to random chance than I had first assumed. Hopefully, the analysts at Gallup and Rasmussen Reports, who can run tests using the independent daily samples, will help clarify.
One more thing: I should be clear that while the charts above came from Hickman and Margolis, the commentary and conclusions were mine and mine alone. So blame me for any confusion.
Update 3: Gallup's Jeff Jones responds with data since January broken out by day-of-week.
Update 4: Harrison Hickman adds his thoughts as well.
Public Policy Polling (D)
Obama 54, Clinton 36
Can you guess which presidential election gave us this headline and lead?:
POLLS: A YEAR TO BE WARY
ONCE in a while, all pollsters should take the kind of beating we took in the primaries, just to maintain equilibrium," says Don Muchmore, board chairman of Opinion Research of California, one of the many polling firms that came a cropper in one or more of this year's presidential primaries.
The phrase "came a cropper" probably gives it away. The year was 1964 when this article in appeared in Time. I was all of 17 months old (my own world had been rocked the previous day, or so I am told, by the birth of my brother...but I digress). In a bit of Internet serendipity, I stumbled on the article yesterday when Googling the name of Oliver Qualyle (Lyndon Johnson's pollster) for the previous item. It is worth reading in full, if only for the perspective it provides on 2008.
It is striking how many of the issues of polling in 1964 -- when most political surveys were conducted in person by interviewers that went door-to-door to select respondents -- remain the same in 2008. The topic of "likely voters" was just a confusing: In 1964, Time reported, pre-election polling techniques "vary considerably from pollster to pollster" and the researchers had "not yet licked" their "inability to assess the probable voter turnout." The Time article also includes a discussion of whether "people lie to the pollsters" on the issue of "race relations" in a year when that issue seemed "particularly important."
On the other hand, on at least one issue, polling has come nearly full circle. In 1964, the concept of random sampling was not yet universally accepted among American pollsters. The article cites a "trend towards 'randomization'" among the pollsters and then, perhaps unknowingly, lists various departures from random sampling in their techniques, such as Gallup's practice, circa '64, of instructing interviewers to "skip some corner houses on the theory that corner property is higher-priced and its occupants are likely to be more affluent than their neighbors."
Today the controversy over random sampling is whether rates of non-response are so high as to render true random sampling impossible, leading some pollsters to return to various departures from true "randomization" such as the use of volunteer online panels.
Back to a polling evergreen. Are polls "reliable enough" to justify their high costs?
In probing general attitudes toward candidates and issues, they undoubtedly come close enough to be of value to campaign strategists. When it comes to calling elections, most of the pollsters insist that they do not make predictions, merely measure the popularity of candidates at a given point in time. In the post-mortems they are, of course, the first to boast when they hit one right. But that seems fair enough, since they take a beating when they are wrong.
By the way, the entire Time archive of articles published since 1923 is now available online to non-subscribers. Happy searching.
American Research Group
Obama 51, Clinton 38
A bit of polling history on a slow Monday morning: Yesterday's Los Angeles Times included a fascinating account of President Lyndon Johnson's decision against seeking a second term in 1968 by his former appointments secretary Jim Jones (via Jonathan Martin). The story includes reference to polling done for Johnson in New Hampshire:
As appointments secretary to Johnson -- the position now known as chief of staff -- I followed the ups and downs of the president's decision-making process closely, and I am convinced that fear of losing was not why he declined to run. In fact, just prior to his March 31 speech, we instructed our pollster, Oliver Quayle, to do an in-depth survey pitting Johnson against all of his competitors in both parties. Johnson defeated every Democrat and Republican candidate by relatively wide margins.
The poll was conducted about the time of the New Hampshire primary. Because Johnson had not said definitively one way or the other whether he would be seeking the nomination, insurgent Minnesota Sen. Eugene McCarthy was the only serious candidate on the Democratic ballot. Even though McCarthy did much better than anyone expected, the fact is that Johnson won the primary with 49% of the vote -- all on write-ins.
In 1986, political scientist Bruce Altschuler published a review of "Lyndon Johnson and the Public Polls" ($) in Public Opinion Quarterly. The account makes no mention of this final New Hampshire poll, although it identifies Quayle as "Johnson's main pollster," someone who "regularly took polls, usually of individual states or parts of states between 1964 and 1968.