July 2, 2009
By Mark Blumenthal
The Kaiser Family Foundation rounds up polling on how to foot the bill for health care reform.
Scott Keeter and his Pew Research Center colleagues review the perils of polling in election 2008.
Stu Rothenberg reviews the perils of reporting on polls.
Carl Bialik considers the limitations of Benford's Law in hunting vote fraud (more on his blog).
Andrew Gelman shares his final thoughts on the Iran vote analyses.
Alex Bratty sees signs of Obama buyer's remorse among independents.
Gary Andres says perceptions of risks will matter more than perceived benefits in health care reform.
Ruy Teixeira argues that Republicans are out of touch on health care.
Mark Mellman says Republicans will pay for opposing clean energy.
David Hill likes contested primaries.
Ben Tulchin sees Gavin Newsome gaining after Antonio Villaraigosa drops out of the CA governor's race.
Greg Sargent notes Republican division on Sotomayor.
Tom Jensen thinks automated surveys are picking up more Sotomayor opposition.
Chris Weigant updates his Obama Poll Watch charts.
John Sides faults Charles Blow for succumbing to an ecological fallacy.
Haaretz reports a new poll showing Hamas popularity falling among Palestinians (via Crowley).
Research Rants reminds web researchers to test their surveys.
Gary Locke taps three Census Bureau veterans to serve as part time advisors.
Survey Practice releases its June issue -- so you can spend the long weekend catching up on the optimal number of scale points for attitudinal questions.
And an Andy Borowitz parody comes disturbingly close to the truth.
By Mark Blumenthal on July 2, 2009 4:28 PM
| Permalink
| Comments (0)
| TrackBacks (0)
July 1, 2009
By Mark Blumenthal
Today brings two new national poll releases featuring in-depth questions on health care reform from CNN/ORC and Quinnipiac University. As always with the subject, the new releases provide many new wrinkles to consider, but for the moment I want to focus on just one.
The CNN/ORC poll begins with a very general measure: "From everything you have heard or read so far, do you favor or oppose Barack Obama's plan to reform health care?" They find a "slim majority" (51%) in favor, 45% opposed and 4% unsure.
Let's start with what is hopefully obvious: Democrats in Congress are drafting multiple proposals, and the Obama administration has not specifically endorsed any of these. So a well informed respondent ought to have trouble evaluating "Obama's plan," since Obama has not yet committed to a specific plan. Even more important, very few Americans are following that debate with rapt attention. Last month's CBS/New York Times poll, for example, found only 22% of Americans saying they have heard or read "a lot" about the health care reform proposals (50% said they heard or read "some," 23% not much, 5% nothing).
Notice that when the NBC/Wall Street Journal poll asked a general question about "Obama's health care plan" last month, they offered "no opinion" as an option: "From what you have heard about Barack Obama's health care plan, do you think his plan is a good idea or a bad idea? If you do not have an opinion either way, please just say so."With that option, slightly more than a third (35%) either had no opinion or were unsure. Those with an opinion divided evenly; 33% said it was a good idea, 32% a bad idea.
When pollsters push as hard as CNN/ORC for an answer, a lot of the responses are going to be very soft, often formed on the spot and based on very superficial impressions. Nonetheless, if I were charged with conducting a benchmark survey for a candidate over the next few months, and I had room for only one question about health care reform, I would be tempted to ask a very general question about "President Obama's plan to reform health care" (though I'd strongly lean to the NBC/WSJ version that explicitly prompts for "no opinion").
Yes, public opinion on health care reform is multi-faceted. Americans come to the debate with a rich set of values and attitudes about what they like and dislike about the health care system, what they would change and what they worry about changing. Most have not yet focused on the details of the legislative debate. Many never will. So questions about specific policy proposals can produce results all over the map. As Slate's Chris Beam puts in an excellent summary this week, "health care polling is especially variable, depending on the wording, the context, and the momentary angle of the sun."
But once a specific health care reform proposal comes up for a vote, members of Congress are going to be intensely interested in the bottom line perceived by their constituents: Do they generally favor or oppose the thing they are about to vote on? Right now, many Americans do not hold strong opinions about whatever they think "Barack Obama's health care plan" is or will be, but those attitudes are likely to deepen and change in the coming months.
In that regard, the comparison provided by CNN Polling Director Keating Holland is helpful:
"In September of 1993, when Bill Clinton was just starting to roll out his ill-fated health care plan, 54 percent said they supported Clinton's ideas on that issue. Today, 51 percent feel the same way about Obama's proposals," Holland said. "That indicates that Obama may have his work cut out for him in the coming months."
CNN also sent out a release this afternoon that includes the complete time series of this question from the 1993-1994 period. I used it to create the following chart:
Support and opposition is roughly comparable what the CNN/USA Today/Gallup polling partnership found in early 1993, although note that their first two surveys in September 1993 came just before and just after President Clinton delivered a live, prime-time address outlining the specifics of his proposal to a joint session of Congress. Note also that the average "unsure" percentage on the 1993 surveys was 9%, slightly more than double the 4% on the survey released today. CNN's surveys are now fielded by the Opinion Research Corporation (ORC) rather than Gallup, and ORC's interviewers may be pushing slightly harder for answers than Gallup's interviewers were 16 years ago.
Caveats aside, this is a measure worth watching, but be careful to keep it in context: Nearly a third of Americans, when offered the option, say they have no opinion (yet) of Barack Obama's "plans to reform health care."
Update: Nate Silver has more.
By Mark Blumenthal on July 1, 2009 2:57 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
Too good to hold for the next "outliers" feature: Monday night's Colbert Report had a segment on the U.S. Census that even includes a reference on on-hold Census director nominee Robert Groves (via @AAPOR).
By Mark Blumenthal on July 1, 2009 10:35 AM
| Permalink
| Comments (1)
| TrackBacks (0)
June 30, 2009
By Mark Blumenthal
Another piece of housekeeping and one of Eric Dienstfrey's final contributions to Pollster.com. We have produced a new chart that includes only polls that report the Obama job rating among all adults. The original Obama job rating chart that includes all surveys remains in place; this new chart adds a new way of tracking the trends.
We have discussed some of the challenges posed to our charts on measures like the Obama job performance rating by pollsters whose results show big "house effects" (consistent differences when compared to other pollsters). Our philosophy has always been to try to include all polls that claim to produce representative samples -- even those based on more controversial methods such as automated polls or those that survey respondents over the internet using pre-recruited panels -- to make it possible to use our interactive chart features to compare and contrast different surveys.
The problem is that if big house effects occur, the trend lines can sometimes display phantom trends when polls with consistently different results are more frequent. This issue crops up most often in the "nose" of the trend line, which moves around more than the rest of the line as we add new polls to our database. The Rasmussen Reports surveys appear to be a big problem in this respect, mostly because they are far more numerous. However, if you use your mouse to click on Obama job ratings that tend to be higher or lower than other polls, you will also see pollsters with similar house effects that poll less often.
Chart With All Surveys:
We offer the new all-adult-sample-only charts as one means of reducing the potential for "phantom" trends, though we have other potential improvements in the works. Please let us know what you think.
PS: A week or so ago we also broke out party identification in two: one is based on results among all adults, one among surveys of registered or likely voters.
By Mark Blumenthal on June 30, 2009 4:02 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
Regular readers have probably noticed a new name appearing on the "poll update" entries on Pollster.com. Emily Swanson, a recent graduate of the University of Wisconsin-Madison, has joined the Pollster.com team and will be posting and updating our charts and tables regularly from here on out. Welcome Emily!
Unfortunately, Emily's appearance means that we are saying farewell to Eric Dienstfrey after nearly three years of relentless hard work and dedicated service. As announced a few months ago, Eric has been accepted to the Graduate Program in Film Studies at, coincidentally, the University of Wisconsin-Madison's Department of Communication Arts. So he is moving on to bigger and better things.
Sadly, today is officially Eric's last day at Pollster.com. I exaggerate not one bit when I say that the site as you know it would not exist but for his skill and tenacity. We will miss him, but wish him the best of luck in all of his future endeavors.
By Mark Blumenthal on June 30, 2009 12:03 PM
| Permalink
| Comments (0)
| TrackBacks (0)
June 29, 2009
By Mark Blumenthal
So what has higher customer satisfaction. private health insurance plans or the Medicare program? The answer, revealed in my NationalJournal.com column for the week, may surprise you.
Some additional details that were a bit too wonky for the column: I cite results from surveys conducted using a standard questionnaire developed by a the Consumer Assessment of Healthcare Providers and Systems program, known better by its acronym, CAHPS (pronounced "caps"). The program is an initiative of the U.S. Agency for Healthcare Research and Quality (AHRQ) of the Department of Health and Human Services to create a standardized survey for patient satisfaction that could be used by health insurance plans and hospitals (and, full disclosure, my wife used to work at AHRQ, though she was not directly involved with the CAHPS program).
Why would the government care about creating a standardized "customer satisfaction" questionnaire for health care? In the late 1990s, Congress allowed states to enroll Medicaid recipients into managed care plans run by private insurance companies, but the states had to offer multiple plans. One of the reasons for the CAHPS initiative was to create a standard for providing uniform quality information to Medicaid recipients.
The non-profit National Committee for Quality Assurance (NCQA) also played a significant role in the CAHPS' creation. NCQA uses CAHPS data to evaluate and accredit private health plans, so those companies that wish to receive NCQA's seal of approval or rank high in the U.S. News & World Report annual feature on "America's Best Health Insurance Plans" have an incentive to participate. As a result, roughly 90 percent of private insurers conduct CAHPS surveys and provide the data to NCQA.
Survey reachers of all stripes can learn from the extensive research and development that went into the creation of CAHPS program. One of its goals was to create a survey questionnaire based on the "best science...the state-of-the-art in survey and report design," and there are few precedents for what they achieved. In its first phase, the CAHPS program spent over $5 million on cognitive pre-testing and other pilot studies of a questionnaire developed jointly by RTI, RAND, Harvard Medical School & Westat. The lessons they learned should be of interest to anyone conducting a customer satisfaction survey.
Most of the CAHPS data I cited in the column come from a Health Plan Survey Chartbook published by AHRQ that includes compilations of data culled from over hundreds of individual surveys and literally hundreds of thousands of interviews conducted with patients in private insurance plans and managed care plans that enroll Medicare and Medicaid beneficiaries. The comparison data for those in traditional, fee-for-service Medicare comes from a spokesperson for the Centers for Medicare & Medicaid Services (CMS). Note that although the AHRQ chart book includes more recent data, I cited data from 2007 that would be comparable to the survey data on fee-for-service Medicare recipients that I received from CMS.
By Mark Blumenthal on June 29, 2009 11:49 AM
| Permalink
| Comments (0)
| TrackBacks (0)
June 26, 2009
By Mark Blumenthal
The Pew Research Center uses fewer words, gets a different result on spending vs. deficit reduction; Ed Kilgore, and Linda Hirshman react.
Gallup finds 92% consider extramarital affairs immoral.
Nate Silver compares calls on Sanford to resign to those faced by other "pantsless pols."
Jennifer Agiesta notes a cooling among liberals toward Obama on global warming.
National Journal's insiders see a rough road for Cap and Trade.
David Hill considers why choice, competition and a plan "like Medicare" might appeal to Republicans.
Matthew Cooper has a theory on Obama's slowly declining job rating.
Carl Bialik chalks up a success for automated surveys in Virginia.
Tom Jensen sees a correlation between PPPs Obama job rating and his 2008 vote.
David Winston tests the effectiveness of GOP attacks on Nancy Pelosi.
Greg Sargent reports that Democrats are attacking Republicans for their low poll numbers.
Mark Mellman critiques the poll reporting of the London Independent.
Brenden Nyhan says its wrong to assume only Republicans mistake Obama for a Muslim.
Chris Bowers assesses Sestak vs. Specter.
Michelle Bachmann prefers ignorance to an accurate Census.
By Mark Blumenthal on June 26, 2009 11:09 PM
| Permalink
| Comments (0)
| TrackBacks (0)
June 25, 2009
By Mark Blumenthal
I want to follow-up on yesterday's post about satisfaction with health care coverage and cost, by making an admittedly wonky methodological point that yields a lesson about what makes Americans both eager for health care reform and nervous about it.
Before I started blogging I had the good fortune to conduct a long-running customer satisfaction survey program for a major American corporation. The heart of the survey was a battery of questions that asked customers to rate their satisfaction with various aspects of the company's service. Satisfaction questions of this sort have long been a staple of market research, so I never had any hesitation about asking them, but over the course of that survey are grew more and more, well, unsatisfied with questions that ask about "satisfaction," per se.
The biggest problem is that satisfaction is an attitude based on a comparison between expectations and experience. You might express satisfaction, for example, with a less than optimal experience if you start out with low expectation for that service. I wonder if something like that may be happening with satisfaction questions pertaining to the cost and quality of health care coverage.
Consider the results obtained in surveys conducted by Democracy Corps, the polling project run by Democratic pollster Stan Greenberg and Democratic consultant James Carville. On a survey they conducted earlier this week, 71% of voters say they are satisfied with their "own health insurance coverage" (44% are very satisfied), while 25% are dissatisfied (and note, they asked this question of voters with and without health insurance). But in their analysis, Democracy Corps reports:
Conservatives and some in the media think these voters are not serious about change, but that misreads them, as we realize from our focus groups last week. They are "satisfied" with their choice of doctors, that their employer is picking up most of the cost and that they may have better insurance than others. But, they are not happy about having traded off wages or gotten locked into a job because of health care or about the fate of a child with a chronic ailment who may not be able to get insurance in the future. So, they are nervous about change, but they want it.
Greenberg and his colleagues argue that the sense of "satisfaction" with current coverage may hide negative experiences that make Americans uncomfortable with the status quo. And some findings from other surveys bear them out. In addition to the Kaiser Family Foundation results I cited yesterday, consider this example from the June survey by the Pew Research Center. They found nearly half of adults (48%) saying that "paying for the cost of a major illness" is a "major problem for you and your family" (50% say it is "not a problem"). Almost as many (43%) say that "paying for the cost of health insurance" is a major problem (61% say it is not).
And even though Americans are satisfied with their current coverage, they also express great anxiety about the future. The CBS/New York Times survey, for example, found 49% of Americans saying they are very concerned, and 37% somewhat concerned "about the health care costs you and your family might face in the coming years" (only 13% were not at all concerned). The ABC/ Washington Post survey obtained a similar result: "A whopping 85 percent are concerned about their future costs, with 59 percent 'very' concerned."
Of course, that same ABC/Post surveys shows that the anxiety cuts both ways: "About eight in 10 [adults]," the ABC analysis tells us, "are concerned that reform may reduce their quality, coverage and choice of care, and increase their costs, government bureaucracy and the federal deficit, with anywhere from 51 to 62 percent 'very' worried about each of these."
Yes, in this climate, as KFF's Mollyann Brody told me, "it is really easy to scare people into thinking that reform will make their own situations worse off." But at the same time, people are also very anxious about their costs and future coverage under the status quo. It is that latter anxiety -- much less than any altruistic desire to help out Americans without health care coverage -- that drives the huge general desire for change and reform.
By Mark Blumenthal on June 25, 2009 3:25 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
CBS News Director of Surveys Sarah Dutton and CBS consultant (and former polling director) Kathy Frankovic answer critics who claim their most recent survey "skews left" because a question about whether respondents supported Obama or McCain in 2008 shows Obama winning by a much wider margin (48% to 25% among all adults) than he actually received (53% to 46%). The fact that the answers to a retrospective vote question fail to match the actual vote, they write, "is nothing new:"
This poll and others have asked about past vote for many years, and the results rarely match the voting results.
Here's the most extreme example: The University of Michigan's National Election studies, arguably the best academic analyses of why Americans vote the way they do, found huge disparities in how Americans reported their 1960 vote in their 1962 and 1964 post-election studies. When Americans were asked just after the 1960 election how they voted, John Kennedy and Richard Nixon were tied in the poll (matching the results: Kennedy just eked out his 1960 victory). Two years later, Americans recalled giving Kennedy a comfortable win. In 1964, after Kennedy was assassinated, the difference was even more dramatic - Americans "remembered" that they had elected Kennedy by a two to one margin!
It goes in both directions - Democratic AND Republican: In January 2002, when George W. Bush's approval rating was 82 percent, Americans recalled that they had given him a 12-point victory over Al Gore in November 2000.
Read the rest for their explanation for this common occurrence.
Slate's Chris Beam wrote about this topic last week; I wrote about it four years ago.
[Correction: The original version of this post failed to identify Kathy Frankovic as a co-author].
By Mark Blumenthal on June 25, 2009 11:27 AM
| Permalink
| Comments (0)
| TrackBacks (0)
June 24, 2009
By Mark Blumenthal
A follow-up to my post two weeks ago on the interactions my father-in-law and family had with considerate and professional staff of the U.S. Holocaust Memorial Museum. The impetus for that post, of course, was the murder of Officer Stephen Tyrone Johns as he stood guard at the Museum.
Given the outpouring of kind words I received to that post, I thought readers would want to know that in order to assist the Johns family, the Museum has established the USHMM Officer Johns Family Fund and is accepting donations on its behalf. They pledge that one hundred percent of your gift will be forwarded directly to the family of Officer Johns, but note that under IRS regulations gifts such as this are not tax-deductible.
You can donate online, by telephone or via U.S. mail:
Checks payable to USHMM Officer Johns Family Fund may be mailed to USHMM, 100 Raoul Wallenberg Place, SW, Washington DC 20024.
You may also make a gift by calling toll free 877-91USHMM (877-918-7466) from 8:00am to 8:00pm Eastern time.
By Mark Blumenthal on June 24, 2009 9:34 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
The ABC News analysis of the newly released ABC/Washington Post poll results on health care reform leads off with this paragraph:
The chief obstacle to reform is that large majorities are satisfied with their current care and coverage; most, albeit fewer, also call their costs tolerable. Dissatisfaction with the system overall, and worry about future costs, are countered by broad concerns that change could worsen the quality, choice and coverage most Americans enjoy now.
The result: pushback works.
While I have no quarrel with that summary -- my column earlier this week made a similar point -- it is easy to miss the part about "tolerable" costs and the worry about costs in the future. So for this post, I want to focus more on those measures of satisfaction, especially those pertaining to cost. (I will leave questions about specific policy options for another post, although, as Nate Silver points out this morning, it is also possible to "push" respondents in the direction of greater support for reform).
The ABC/Post poll asked four satisfaction questions about health care and health coverage. These illustrate the point in the ABC analysis: Americans are generally satisfied with the quality of their health care, but less so their coverage and much less so with cost, although they still find more Americans satisfied (54%) with "health care costs, including both expenses not covered by insurance, and the cost of your insurance" than dissatisfied (44%).
A minor nitpick: The results reported on the ABC/Post poll for "your health insurance coverage" were tabulated among those with health coverage. That's fine, but in comparing across items we might want to factor in the 16% that say (on this survey) that they lack insurance coverage. Do that and we find that 68% are satisfied -- still a big number -- 16% are dissatisfied and 16% lack insurance.
Also note that while 49% of Americans are "very satisfied" with the "quality of care" they receive, only 35% of Americans have coverage and are very satisfied with it. Put another way, it looks as though nearly half either lack insurance coverage, are dissatisfied with their coverage or are less satisfied with their coverage than they are with the quality of their care. The point is, Americans differentiate between the quality of their care and the quality of their coverage.
Now take a look at a very similar set of satisfaction questions asked by CNN/ORC back in March. They asked very similar questions about the quality of health care received, insurance coverage and "your health care costs" and obtained virtually identical results (and it looks as though they asked their coverage question of respondents without coverage -- a few, presumably, were satisfied with their lack of coverage).
What catches my eye in this context, however, is the probe of satisfaction with "the total cost of health care in this country." On that measure, Americans are far less satisfied (23%) than on the ABC/Post question about "the overall health care system in this country" (42%). So guess what worry appears to be driving the desire for reform? Cost. Or more precisely, Americans worry about rising costs and how they may affect their access to needed care in the future.
The April tracking survey conducted by the Kaiser Family Foundation took a more extensive look how Americans experience and perceive health care costs. They found 72% of Americans worried about "having to pay more for your health care or health insurance" (37% were very worried). Almost as many (66%) said they worried about "not being able to afford the health care services you think you need" (34% were very worried. They also found cost concerns affecting the treatment that Americans seek:
As the economy continues to falter, a majority of Americans continue to say they or someone in their household have taken steps to put off health care for cost reasons over the course of the last year. Overall, six in ten (59 percent) say they have taken at least one of seven steps to delay or skip care this past year
According to a new Kaiser Family Foundation health tracking survey, most common was relying on home remedies or over the counter drugs rather than consulting a physician, which 42 percent report, followed by skipping dental care. Three in ten reported not filling a prescription.
So yes, Americans are generally satisfied with the quality of care they receive, but less satisfied with their coverage and especially the cost of that coverage and other necessary medical expenses they incur. They also worry a lot about about being able to afford the health care services they might need in the future. Those are the attitudes that fuel the desire for reform.
By Mark Blumenthal on June 24, 2009 1:49 PM
| Permalink
| Comments (0)
| TrackBacks (0)
June 23, 2009
By Mark Blumenthal
Gallup now publishes weekly demographic crosstabs for presidential approval (downloadable spreadsheet too!).
Jennifer Agiesta shares the Washington Post/ABC job approval demographic cross-tabs.
John Hinderaker says the CBS/New York Times health reform survey "skews left," Steve Benen and Eric Kleefeld counter (see also Chris Beam and yours truly).
More on the CBS/NYT health care reform survey from Gary Andres, Steve Benen, John Graham, Ed Kilgore, Nolan McCarty, McJoan, Ramesh Ponnuru, Greg Sargent and Josh Tucker
Nate Silver reviews multiple polls and their results on the public option.
Glen Bolger explores the Resurgent Republic health care polling data.
Andrew Gelman points to some statistical methods for election auditing.
Michael Franc sees a new conservative plurality.
Sheri and Alan Rivlan call for compromise to pass health care reform.
Ben Sargent notices an increase in GOP negatives in the wake of the Sotomayor nomination.
Jay Cost thinks we make too much of presidential "teflon."
DemFromCt gathers the low approval ratings of Republicans from multiple polls.
Tom Jensen ponders divergent poll results in Ohio.
Think Progress unmasks a "progressive" anti-immigration reform polling effort (via @CenteredPols)
Arlen Specter hires Democratic pollsters Jef Pollock and Mark Mellman
Nancy Scola highlights the powerful combination of Google search, Data.gov and Gapminder visualization.
By Mark Blumenthal on June 23, 2009 3:58 PM
| Permalink
| Comments (1)
| TrackBacks (0)
By Mark Blumenthal
Sad news from Gallup editor-in-chief Frank Newport:
Alec Gallup, one of the polling world's most committed practitioners and dedicated supporters of the value of polling and all around good guys passed away last night. Alec was one of two sons of Dr. George Gallup and was the long time Chairman of the Gallup Poll. Alec lived in Princeton, New Jersey. Anyone who has worked at or with the Gallup Organization over the years and who came into contact with Alec recognized what a truly unique individual he was. He literally devoted all of his life to polling -- spanning his childhood days when he worked with his father as poll "ballots" came in via train to be tabulated at Gallup headquarters up to as recently as a week or two ago, when, even in declining health, he would call up and make suggestions about what poll questions Gallup should be asking in the current political environment. Polling has never had a greater champion, and those who knew Alec personally have never had a greater friend. Everyone who knew Alec will miss him immensely.
Alec Gallup was interviewed about his father and the early days of polling nine years ago for a PBS documentary. You can read a transcript here (via Mike Mokrzycki).
By Mark Blumenthal on June 23, 2009 1:16 PM
| Permalink
| Comments (0)
| TrackBacks (0)
June 22, 2009
By Mark Blumenthal
Joe Lenski, whose Edison Research has been the sole provider of exit polling to the U.S. television networks since 2003, recounts his experience conducting an exit poll for the Azerbaijan parliamentary elections in 2005. He sees some significant parallels between Azerbaijan and the election just held in Iran:
First, the [Azerbaijan] election and the voting itself, which seemed open and fair at the time, turned out to be mostly for show. Once the initial results in Azerbaijan showed opposition candidates winning in certain districts, the electoral commission took steps to make sure the official results matched what the government desired, which is probably what happened in Iran. Basically, the authorities were fine with an open and seemingly fair election as long as their guy won. Once that outcome became less certain, procedures were probably put in place to make sure the official results matched what the government desired. The fraud is probably not universal, but, as in Azerbaijan, concentrated in the areas where the opposition was doing better than expected. This would explain much of the statistical evidence showing Ahmadinejad doing well in the official vote returns from provinces and towns where the reformist candidates did well in the 2005 election.
He has much more. Read the whole thing.
By Mark Blumenthal on June 22, 2009 7:53 PM
| Permalink
| Comments (1)
| TrackBacks (0)
By Mark Blumenthal
Walter Mebane has once again updated his analysis of the official vote count in Iran (via Gelman), having now obtained ballot box data in 23 of 30 provinces. His modified conclusion (raw data here):
The initially released polling station data show evidence of significant distortions in the vote counts not only for Karroubi and Rezaei but also for Ahmadinejad. No significant distortions are apparent for Mousavi's vote counts. A key to interpreting these results is understanding why the vote counts for Karroubi and Rezaei are typically so small. Is it (a) inherently low levels of support, (b) voters strategically abandoning the candidates, or (c) fraudulent counts? If there is good reason to believe either (a) or (b), then (c) is less likely. The significant result for Ahmadinejad is not direct proof that Ahmadinejad's votes are fraudulent, but fraud is certainly a reasonable inference in light of reports that "Iran's Guardian Council has admitted that the number of votes collected in 50 cities surpass the number of those eligible to cast ballot in those areas" (Press TV, 2009).
Andrew Gelman, who is "inclined to believe" Mebane because he is "the expert on this stuff" (I had a similar message on Thursday) also repeats this important caveat, which is consistent with Mebane's own conclusions:
[T]his sort of statistical analysis doesn't prove anything by itself, but it can be useful in giving people a sense of where to focus attention if they want to look further.
By Mark Blumenthal on June 22, 2009 7:44 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
In addition to the turmoil and tragedy in Iran over the weekend, there were two new notable analyses of the official turnout, plus one bizarre concession by the ruling Guardian Council. Let's start with a review of the analyses:
- Last week, we pointed to an analysis (pdf) by American political scientist Walter Mebane (explained further here). He used the county and city-level vote data from the two rounds of Iran's 2005 election to try to model the 2009 result. The underlying idea is to see whether the town-by-town variation in Ahmadenijad's vote in 2005 predicts the town-by-town variation in 2009. He found that his model did not "describe" the vote well in 192 of 320 towns and that, in 172 of those, Ahmadenijad's vote looks suspiciously high. [Update: Mebane has updated his analysis based on new ballot box data for 23 of 30 provinces showing "evidence of significant distortions in the vote counts not only for Karroubi and Rezaei but also for Ahmadinejad" - more here].
- Over the weekend, Alex Scacco and Bernd Beber, graduate students at Columbia University,** published analyses in the last two digits in reported 2009 vote totals, on the theory that the distribution of these digits should be totally random. The found suspicious patterns suggestive of fraud in the provincial-level data but not in what they describe as county-level data. Their theory is that provincial level data were fabricated and that the "leading digits" of the county-level data subsequently manipulated to match fraudulent provincial totals (which would have required minimal tampering with the last two digits of most counties -- R analysis code and data here, via Monkey Cage).
- Yesterday, the British think-tank Chatham House published an analysis of the provincial level data co-authored by academics at the Institute of Iranian Studies at the University of St. Andrews. They found irregularities in turnout -- including two provinces showing "a turnout of over 100%" -- and patterns they found implausible in the supposedly new votes cast for Ahmadenijad in 2009. Note that while the Mebane and Scacco-Beber analyses were mostly statistical, the Chatam House analysis is more steeped in the authors' expertise in recent Iranian political history.
But perhaps most telling was this statement yesterday from the Iran's ruling Guardian Council yesterday as published by Iranian state television:
Iran's Guardian Council has suggested that the number of votes collected in 50 cities surpass the number of people eligible to cast ballot in those areas.
The council's Spokesman Abbas-Ali Kadkhodaei, who was speaking on the Islamic Republic of Iran Broadcasting (IRIB) Channel 2 on Sunday, made the remarks in response to complaints filed by Mohsen Rezaei -- a defeated candidate in the June 12 Presidential election.
"Statistics provided by the candidates, who claim more than 100% of those eligible have cast their ballot in 80-170 cities are not accurate -- the incident has happened in only 50 cities," Kadkhodaei said.
Kadkhodaei further explained that the voter turnout of above 100% in some cities is a normal phenomenon because there is no legal limitation for people to vote for the presidential elections in another city or province to which people often travel or commute.
To put this in perspective, that's 50 of over 300 cities in which turnout exceeded 100% of the eligible voters. So the statement truly pushes the boundaries of "spin," or as Nate Silver puts it, "Worst. Damage Control. Ever." Nate says they are admitting to "some fraud" just not "11 million votes worth of fraud," though I'm not sure I would go that far. Kadkhodaei claims that the pattern is a "normal phenomenon," since it is legal for Iranian's to vote outside their home provinces. Still, it's quite a stretch.
Consider the update from the Chatam House authors (see p. 2) that their "results are not significantly affected" by the Guardian Council statement:
Whilst it is possible for large numbers of voters to cast their ballots outside their home district (one of 366), the proportion of people who would have cast their votes outside their home province is much smaller, as the 30 provinces are too large for effective commuting across borders. In Yazd, for example, where turnout was above 100% at provincial level, there are no significant population centres near provincial boundaries.
Note also that they found, separately, that the increase in turnout in 2009 "results in substantially less variation in turnout between provinces, with the standard deviation amongst provincial turnouts falling by just over 23% since 2005." So the Guardian Council's argument is that out-of-province voting was great enough to cause turnout beyond 100% of eligibility in 50 towns and 2 provinces, yet the Chatham House analysis shows less variability across provinces than in 2005. That's quite a pattern.
Update: Josh Tucker has more on the Kadkhodaei statement.
**The original version of this post misstated Scacco and Beber's academic affiliation - see the comment from Andrew Therriault.
By Mark Blumenthal on June 22, 2009 3:19 PM
| Permalink
| Comments (2)
| TrackBacks (0)
By Mark Blumenthal
My NationalJournal.com column for this week, on the deluge of new polls on health care reform and how to make sense of them, is now posted. Since filing on Friday, The New York Times and CBS News released a new set of results on health care reform even more extensive than those posted last week. Their questions bring the "deluge" to over 60 items related to health care reform in the last two weeks. And we are probably just getting warmed up.
By Mark Blumenthal on June 22, 2009 9:06 AM
| Permalink
| Comments (0)
| TrackBacks (0)
June 19, 2009
By Mark Blumenthal
New national polls from CBS/New York Times, NBC/Wall Street Journal and the Pew Research Center inspire commentary and analysis from: Michael Barone, Steve Benen, Alex Bratty, Jon Chait, Chris Cillizza, DemFromCt, John Dickerson, William Galston, Linda Hirshman, Ed Kilgore, Ezra Klein, Taylor Marsh, Neil Newhouse and Jeremy Ruch, Glenn Reynolds, Greg Sargent, Jonathan Singer, Sam Stein, Jake Tapper and Chuck Todd, et. al.,
David Moore provides five reasons to distrust the Terror Free Tomorrow Iran pre-election poll.
John Sides points to a study on whether international observers deter election fraud.
Gallup shows Obama's job approval slipping to 58% for the first time.
Nate Silver looks at health care events correlated with Bill Clinton's job approval in 1993-1994.
Bill Schneider thinks Republicans will have a hard time finding a new leader.
Markos Moulitsas finds greater empathy among Millennials (more here).
Chris Bowers thinks an open Pennsylvania primary would benefit Sestak.
Jennifer Skalka shares her thoughts on the DGA poll of Virginia.
National Journal's insiders grade the White House staff.
Tom Jensen has reason to be irked (or maybe not).
Lee Sigelman summarizes a study on how political consultant compensation affects political campaigns.
Carl Bialik says Nielsen did not provide its panelists with digital converter boxes.
Michelle Bachmann plans to leave most Census questions blank.
The White House wants to ask about same-sex unions in the 2010 Census.
And no, this is not from an Iranian version of Pollster.com. It's a chart (via Sullivan) purporting to show increasing strength of crowds in Iran:
By Mark Blumenthal on June 19, 2009 6:12 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
Slate's Christopher Beam notices an apparently extreme example of something common in political opinion polling. Winning candidates typically do much better in questions that ask respondents to recall who they voted for in the last election:
In the 2008 election, Obama won 53 percent of the votes; John McCain got 46 percent. But two new polls, conducted by the Wall Street Journal/NBC and the New York Times/CBS, show Obama winning by a much wider margin.
When respondents were asked by the WSJ whom they voted for in the 2008 presidential elections, 41 percent said they voted for Obama, compared with 32 percent for McCain. Factor out the 18 percent who said they didn't vote, and you've got Obama beating McCain by 11 points, 50 percent to 39 percent.
The gap in the New York Times poll is even wider. In it, 48 percent of respondents said they voted for Obama, compared with 25 percent for McCain. Again, subtract the 19 percent who say they didn't vote, and you've got Obama winning by a nearly 2-to-1 margin, with 60 percent to McCain's 32 percent.
This sort of discrepancy is not unusual, as Beam reports and as evidenced by an old MysteryPollster post that he links to. Still, he catches what appears to be an unusually large gap:
[T]he disparity between declared Obama voters and actual Obama voters is especially wide. The gap is usually in the single digits, and it waxes and wanes with the president's popularity. The New York Times poll, conducted periodically since Obama's inauguration, shows the gap between Obama and McCain steadily growing. In February, he led McCain 42 percent to 28 percent. In April, it was 43-25. By June, his lead had grown to 48-25. "Even by the standards of historical numbers, that's a large gap," says Adam Berinsky, a political science professor at the Massachusetts Institute of Technology.
Beam does a good job explaining the likely reasons for the gap, and I recommend reading it in full (though full disclosure: he interviewed me for the piece via email).
By Mark Blumenthal on June 19, 2009 4:24 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
Our discussion of pre-election polling in Iraq (here and here) has so far been limited to the Terror Free Tomorrow/New America Foundation poll. There were other pre-election polls, however. Renard Sexton summarized those results a week ago and found some evidence of momentum toward Mir Hossein Mousavi. Now Sam Wang, the Princeton neuroscientist who has been "meta-analyzing" polling and doing election projections since 2004, has blogged an analysis of polls in Teheran that suggests an "anomaly" with the official count:
National Iranian polls were highly variable and of suspect quality. But within Tehran, polls were more uniform and allow a comparison. Six Tehran polls gave a median lead for Moussavi by 4%. This differs notably from the official tally for the city, Ahmadinejad by 12%. The 16-point discrepancy suggests an anomaly in Tehran and opens the question of whether fraud occurred here - and elsewhere. However, it is also important to note several caveats, including polling uncertainty and possible shifts in opinion following the Ahmadinejad-Moussavi debate on June 3rd.
I want to emphasize Wang's caveats. He limits his analysis to Teheran, and as he notes, even if we assume the 16-point discrepancy observed there applies nationwide, "it would still not be enough to alter the overall outcome." He also points out that we may be mixing geographic definitions: "[The] polls might have been restricted to the actual city of Tehran, which is not all of Tehran province." Still, his analysis adds something worthwhile to the debate.
Update - An alert reader emails:
Seems that Wang's
possible mix-up on the geography is more than a caveat. [In the official count], Mousavi won the
city of Tehran 52-43, while losing the whole province to Ahmadinijad by
six points (51 to 45 percent)
By Mark Blumenthal on June 19, 2009 3:58 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
Walter Mebane, author of the analysis of the 2009 Iran elections that we linked to yesterday, gave a brief interview to the website GOOD. He covers the same ground but with a somewhat easier to follow explanation. The summation:
So when I do the analysis predicting what happened in 2009 based on
the first stage of the 2005 election, I get sort of naturalistic
patterns in the coefficients that represent the model, so to speak,
but the number of outliers explodes. The number of observations [towns or cities] that
the model does not describe goes from 8 to 79. And that's 79 out of
320, that's a large number. And when I look at the towns that are
suspicious, that the model doesn't describe all that well but that I
don't throw out completely, that rises to 192 observations. So out of
320 towns, 192 of them are not well described by the model. Moreover,
in 172 of those, it's Ahmadinejad's vote that looks suspicious. And
among the 172, 119 of those have Ahmadinejad getting more votes than
this natural model predicts.
So now that looks a lot like fraud. He gaining extra votes that
don't match what you predict based on a refined examination of the
previous year's election, taking into account the extra mobilization,
which the model does, and also looking at all four candidates. It's not
a proof by any stretch of the imagination, but it's certainly a more
intuitive explanation to say that there are widespread distortions
where people simply added a lot of votes or did something to augment
Ahmadinejad's support."
By Mark Blumenthal on June 19, 2009 10:41 AM
| Permalink
| Comments (1)
| TrackBacks (0)
June 18, 2009
By Mark Blumenthal
Walter Mebane, the University of Michigan political science and statistics professor who specializes in statistical tools "for detecting anomalies and diagnosing fraud in election results," has updated his assessment of the official vote return statistics for the Iran elections. Mebane now says he sees "moderately strong support for a diagnosis that the 2009 election was afflicted by significant fraud."
In his initial analysis, Mebane used town-level data from the second "run-off" stage of the 2005 Iranian elections to model expectations for the 2009 results. The technical difference in the update is that Mebane has incorporated town-level data from the first stage of the 2005 elections. In his revised analysis, Mebane is struck by "the large number of outliers":
One might expect that given the increased political resolution provided by having measures of the first-stage candidates' support, combined with the turnout ratio variable interactions, the model would do a good job capturing more of the variations in the 2009 vote
His conclusion. Something is fishy in the official 2009 results and the deviations appear to benefit Ahmadinejad:
More than half of the 320 towns included in this part of the analysis exhibit vote totals for Ahmadinejad that are not well described by the natural political processes the model of Table 15 represents. These departures from the model much more often represent additions than declines in the votes reported for Ahmadinejad. Correspondingly the poorly modeled observations much more often represent declines than additions in the votes reported for Mousavi.
Modified conclusion: In general, combining the first-stage 2005 and 2009 data conveys the impression that while natural political processes significantly contributed to the election outcome, outcomes in many towns were produced by very different processes. The natural processes in 2009 Ahmadinejad have him tending to do best in towns where his support in 2005 was highest and tending to do worst in towns where turnout surged the most. But in more than half of the towns where comparisons to the first-stage 2005 results are feasible, Ahmadinejad's vote counts are not at all or only poorly described by the naturalistic model. Much more often than not, these poorly modeled observations have vote counts for Ahmadinejad that are greater than the naturalistic model would imply. While it is not possible given only the current data to say for sure whether this reflects natural complexity in the political processes or artificial manipulations, the numerous outliers comport more with the idea that there was widespread fraud than with the idea that all the departures from the model are benign. Additional information of various kinds can help sort out the question. Remaining is the need to see data at lower levels of aggregation and in general more transparency about how the election was conducted.
Having watched a lot of misleading exit poll pseudoscience ricochet around the internet in the aftermath of our own elections in 2004, I have a reflexive caution about quick blog posts claiming statistical evidence of fraud. Walter Mebane falls in an altogether different category. No one is better qualified to find statistical evidence of fraud in election data. He has made his raw data and R-code available (here), and other statisticians (including those with better knowledge of Iran's elections) may reach different conclusions. But if someone like Walter Mebane is no longer on the fence about the 2009 data, it means a lot.
Update - Washington Post polling director Jon Cohen reminds us of the limits of this sort of circumstantial evidence:
This analysis adds a new dimension to the debate over the results,
but is still well short of "hard evidence" of fraud, particularly given
our limited understanding of voting behavior in Iran.
He also summarizes a Post letter-to-the-editor that makes a point I tried to make (albeit less clearly) on Tuesday:
And all of this may miss a key point brought up by a reader in today's Post: in a letter-to-the-editor, John Cronin of Takoma Park writes that our search for
"proof" through numbers may be misguided. "[W]hen an unelected
ayatollah -- the "supreme leader," no less -- controls much of the
media, the military and the courts, the whole state is effectively
rigged," Cronin writes, "[i]t's hard to imagine any election being
truly fair under such conditions, regardless of the extent to which the
ballot boxes are stuffed."
By Mark Blumenthal on June 18, 2009 10:12 AM
| Permalink
| Comments (2)
| TrackBacks (0)
June 17, 2009
By Mark Blumenthal
Via Sullivan, a new op-ed from Ken Ballen and Patrick Doherty that walks back a little from the nut graph of their original Washington Post op-ed that was the object of much criticism. On Monday they argued:
While Western news reports from Tehran in the days leading up to the voting portrayed an Iranian public enthusiastic about Ahmadinejad's principal opponent, Mir Hossein Mousavi, our scientific sampling from across all 30 of Iran's provinces showed Ahmadinejad well ahead.
Yesterday, on CNN, they qualified:
Our poll concluded three weeks before the election. It does not predict the final vote, nor does it measure a possible surge for Moussavi, which many believe occurred in the final weeks. Instead, as we wrote on Monday, our survey indicates "the possibility that the vote is not the product of widespread fraud" because of Ahmadinejad's formidable early lead.
More importantly, they also review some of the context provided other measures on their survey:
Nearly 80 percent want the right to vote for all their leaders, including the all-powerful supreme leader, while nearly 90 percent chose free elections and a free press as the most important goals they have for their government -- virtually tied with the top priority of improving the Iranian economy.
And here is the most important fact of all: More than 86 percent of those who told us they support Ahmadinejad also choose free elections and a free press as their most important priorities for their leaders. In other words, in our survey, Ahmadinejad supporters back real democratic reforms in Iran as much as supporters of the more avowedly reform candidate Moussavi.
Separately, Mark Mellman joins the chorus of critics of Monday's op-ed, but also makes a similar point about attitudes they measured that were not in sync with Ahmadinejad.
Whether or not "the reelection of President Ahmadinejad is what the Iranian people wanted," [the poll] demonstrates voters dissent from some, though not all, of his polices. In contrast to their candidate, strong majorities support rapprochement with the United States and the establishment of democratic institutions.
While a narrow 52 percent majority supports the development of nuclear weapons, 74 percent would have Iran guarantee not to develop nukes in exchange for trade and investment from abroad. other measures from the first survey:
Of course Mellman also notes some agreement:
[O]ne chilling statistic will send shivers up spines in Jerusalem and elsewhere. Just 27 percent of Iran's voters would favor a peace treaty with Israel if an independent Palestinian state is established, but 62 percent "oppose any peace treaty" and "favor all Muslims continuing to fight until there is no State of Israel in the Middle East." On that issue, Ahmadinejad seems to command a majority.
One reader emailed Monday asking me to urge the sponsors of the survey -- Terror Free Tomorrow (TFT) and the New America Foundation -- to release the raw, respondent level data from their survey in order to enable regional cross-tabulations of their vote preference question. Again, the three-week span between the completion of the survey and Election Day renders the data mostly useless as direct forensic evidence of voter fraud.
However, given that TFT is a non-profit with a stated mission of providing facts to help counter extremism, they would do a real public service by putting the machine-readable, respondent level data online right now. Access to that data would not prove or disprove fraud, but it would allow scholars to get a better understanding of the attitudes present among the Iranian citizenry just before their election campaign got underway in earnest.
P.S. Posting on Tehran Bureau, Muhammad Sahimi makes a similar argument about reform impulses in the Iranian public using the TFT data:
Let us begin with the American poll. According to the poll, 77% of the respondents said that they want the Supreme Leader to be elected directly by the people; 74% favor full inspections of Iran's nuclear facilities to ensure that it will not be used for non-peaceful purposes; 77% favor normal trade with, and full recognition by the United States; 68% favor Iran's government to help the U.S. in Iraq, and 52% favor recognition of Israel in return for U.S. recognition and open. trade. Who espouses such policies? The reformists, not President Ahmadinejad.
90% of the respondents thought that the economy should be the top priority of their government. How has Mr. Ahmadinejad's economic performance been (aside from distributing cash among the poor in the last month of the campaign)? Dismal! Unemployment, inflation, and the costs of housing, fuel, and food have all skyrocketed since 2005
By Mark Blumenthal on June 17, 2009 2:37 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
Texas based Republican pollster David Hill is speaking out in favor of confirming Bob Groves as director of the U.S. Census Bureau and against the mysterious "hold" placed on his nomination by an unnamed Republican Senator. In his weekly column, Hill describes the "furtive opposition" by his own party as "ill-advised" and outlined "a strong Republican case" for Groves' confirmation.
Hill's endorsement of the Groves nomination has some significance, given both the Senate hold and the instantaneous "dismay" that House Republicans expressed about the appointment's supposed "ulterior political agenda."
Tempting as it is to reproduce the entire column, I will confine myself to the two most memorable paragraphs. First, regarding Groves' stellar reputation:
On merit, Bob Groves is an exemplary social scientist. No one is more qualified than he to lead data collection that has such vital implications for commerce and industry, not just political parties. Doesn't Groves's curriculum vitae exude the excellence that Republicans want to bring to governance, especially when the results so profoundly affect entrepreneurship, marketing and business planning?
Second, he provides a personal testimonial:
Groves also has the judgment to handle the job. I once had the good fortune of being a co-consultant with him on a project for the University of Utah. We spent several days there advising that institution's nascent survey research and polling unit. It provided us the opportunity to exchange views about the challenges of polling in a university context. How does the university poll on public policies, topical issues and state elections without interjecting corrosive partisan politics? Bob's counsel was wise and reflected a value-free perspective that will serve him well as Census director. You can be certain that Bob will serve science and the data, not political partisanship. His wholly controversy-free tenure at the University of Michigan's Survey Research Center attests to this conclusion.
I can share a similar story. In the late 1990s, I was lucky to have been a student in two classes Groves taught at the University of Maryland's Joint Program in Survey Methodology (JPSM). At the time, I was very much a partisan Democrat, with a client list that included members of Congress. As an undergraduate, I had professors who took great interest in my political activities and delighted in war stories about the campaign trail (I had taken time off as an undergraduate to work on a presidential campaign). Bob was not among them. If anything, I think he found my political consulting background a bit sleazy.
One of these classes, Introduction to the Federal Statistical System, presented and described the federal statistical agencies (Census, Bureau of Labor Statistics, etc.). We spent a lot of time discussing how these agencies could better fulfill their missions while remaining independent of political pressure. It is the memory of those sessions, more than anything else, that makes me want to laugh out loud at the notion of Groves as a partisan appointee bent on "political manipulation." That is exactly backward. Groves is, as Hill puts it, someone certain to "serve science and the data, not political partisanship."
David Hill deserves a lot of credit for bucking some in his own party by standing up for this nomination -- and do read the whole column to get his complete argument. I hope more Republican pollsters follow his example.
Update: Alex Lundry, of TargetPoint Consulting, joins the Republicans for Groves bandwagon.
Update 2: Not a pollster, but Republican media consultant Mike Murphy says he's "with David Hill."
Update 3: Republican pollster Chris Wilson, founder and CEO of Wilson Research Strategies, adds his name to the list of Groves supporters.
By Mark Blumenthal on June 17, 2009 1:20 PM
| Permalink
| Comments (0)
| TrackBacks (0)
June 16, 2009
By Mark Blumenthal
Stan Greenberg revisits his 1993 health care polling and finds little has changed.
Josh Tucker shares lessons on Iran from his research on the "Colored Revolutions" (see also "Things to Watch" on MonkeyCage).
Renard Sexton reviews whether an Iran recount would address reported irregularities.
John Sides considers the origins of electoral fraud.
Nate Silver looks at the Terror Free Tomorrow/New America Foundation poll; as does Chris Bowers.
Gallup finds "conservatives" are a plurality, draws commentary from Kathryn Jean Lopez, Pete Wehner, Right Wing News (on the Right) Steve Benen and Ed Kilgore (on the Left).
Joe Lenski says Deeds win is a classic outcome in a 3-way primary.
Steve Singiser reviews the polling early line on 2009.
Tom Jensen compares divergent Wisconsin polls, Wisconsin Democrats bash PPP's automated polls.
Americans United for Change creates a cable ad touting a Diageo/Hotline poll result.
DemFromCT shares a health care survey conducted by the Employee Benefits Research Institute.
Businessweek publishes Edward Tufte's slideshow (via Lundry)
The Daily Show reports the fake Iranian exit poll results (worth the click for the "Soccer Imams" reference alone):
And the BBC comedy, Yes, Prime Minister explains how pollsters get the results they want (via @randomsubu):
By Mark Blumenthal on June 16, 2009 4:12 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
Today's Washington Post does a real service with its front page story by Glenn Kessler and Jon Cohen that reviews both the plentiful "signs of fraud" in last week's disputed election in Iran, and the frustrating lack of "hard evidence" to substantiate it. The key paragraph:
There are many signs of manipulation or outright fraud in Iran's disputed election results, according to pollsters and election experts, but the case for a rigged outcome is far from ironclad, making it difficult for the United States and other Western powers to denounce the results as unacceptable. Indeed, there is also evidence that Mahmoud Ahmadinejad, the incumbent president deeply disliked in the West for his promotion of Iran's nuclear program and his anti-Israeli rhetoric, simply won a commanding victory.
The Kessler/Cohen piece is one of the most thorough reviews of what we know and, unfortunately, what we cannot know about whether vote fraud occurred in Iran. The article reviews the inconclusive Terror Free Tomorrow/New America Foundation poll (that Cohen critiqued yesterday) and also goes into more detail on the various "suspicious indicators," including:
- The conflicting claims and counterclaims on election night, and the "relatively consistent" margin for Ahmadinejad as official results were reported.
- The suspicion that "so many ballots were said to have been counted so quickly."
- The apparent secrecy surrounding tallies from individual polling stations and lack of vetting by representatives of opposition candidates.
- The questionable pattern of regional results, including surprisingly poor performances by opposition candidates Mousavi and Karroubi in their home provinces.
The bottom line?
"There are suspicious elements here, but there's no solid evidence of fraud," said Walter R. Mebane Jr., a University of Michigan professor of political science and statistics and an expert on detecting electoral fraud.
Separately, as noted by Andrew Sullivan via the blog Stochastic Democracy, Mebane has produced a brief report with the details behind his conclusions. He compares "district level vote counts" and turnout in 2005 to predict support for Ahmedinejad in 2009. He reports that his model works well for most districts, but those deviations from the model he observed were in Ahmedinejad's favor [Correction: Both Stochastic Democracy and I misread Mebane. The the outliers from Mebane's model show places where Ahmadinejad's vote share is smaller than expected, not larger]. Mebane's conclusion:
In general, combining the 2005 and 2009 data conveys the impression that a substantial core of the 2009 results reflected natural political processes. In 2009 Ahmadinejad tended to do best in towns where his support in 2005 was highest, and he tended to do worst in towns where turnout surged the most. These natural aspects of the election results stand in contrast to the unusual pattern in which all of the notable discrepancies between the support Ahmadinejad actually received and the support the model predicts are always negative. This pattern needs to be explained before one can have confidence that natural election processes were not supplemented with artificial manipulations. Also remaining is the need to see data at lower levels of aggregation and in general more transparency about how the election was conducted.
[Update: Mebane subsequently updated his analysis with further data and found "moderately strong support" for fraud].
It is worth remembering that truly "solid evidence" is hard to come by given the closed nature of both the Iranian election and its government. Consider the proof offered in the preliminary report issued on the disputed Ukranian election in 2004 by the most authoritative monitor, the Office of Security and Cooperation in Europe (OSCE). They found "significant shortcomings" in the Ukraine election based on eyewitness testimony and physical evidence gathered by the 4,000 international observers, plus 10,000 more from within Ukraine deployed as accredited journalists. As far as I know, no such independent monitoring occurred in Iran.
It is also worth asking whether, at a certain level, we need to search for "proof" of fraud in the patterns of official vote reports given the graphic and indisputable evidence on display over the last few days in Iran of a ruthless, violent suppression of fundamental human rights. View this extremely disturbing video that Sullivan linked to (if you can bear it) of black-masked police mercilessly beating a defenseless protester, at one point apparently attempting to break his bones while shouting to the crowd, "watch this."** Evidence of fascism may not equate to evidence of vote fraud, but...does it matter?
**Update: The video apparently dates back two years, although it's probably worth asking the same question. Does it matter? Watch Sullivan's blog for similar video from the last few days.
By Mark Blumenthal on June 16, 2009 3:10 PM
| Permalink
| Comments (1)
| TrackBacks (0)
June 15, 2009
By Mark Blumenthal
Today's Washington Post features an op-ed by Ken Ballen and Patrick Doherty that cites a "scientific" survey they helped conduct of the Iranian people "from across all 30 of Iran's provinces [that] showed [President Mahmoud] Ahmadinejad well ahead." Ballen and Doherty argue that rather than indicating "fraud and electoral manipulation...The fact may simply be that the reelection of President Ahmadinejad is what the Iranian people wanted."
This morning, Washington Post polling director Jon Cohen blogged a helpful reality check:
[A] closer look at the one sponsored by Terror Free Tomorrow and the New America Foundation [cited by Ballen and Doherty] reveals ample reason to be skeptical the conclusions drawn from it.
Methodologically, this survey passes muster as it's relatively straightforward to pull a good sample of the Iranian population, using the country's publicly available population counts and listed telephone exchanges. But the poll was conducted from May 11 to 20, well before the spike in support for Mousavi his supporters claim.
(See here for a summary of available Iran polls that finds some evidence for Mousavi momentum late in the campaign.)
More to the point, however, the poll that appears in today's op-ed shows a 2 to 1 lead in the thinnest sense: 34 percent of those polled said they'd vote for Ahmadinejad, 14 percent for Mousavi. That leaves 52 percent unaccounted for. In all, 27 percent expressed no opinion in the election, and another 15 percent refused to answer the question at all. Six percent said they'd vote for none of the listed candidates; the rest for minor candidates.
The whole thing is worth reading in full.
Update: Gary Langer, polling director of ABC News, covers much of the same ground, but catches that Terror Free Tomorrow's pre-election analysis "predicted a runoff":
To declare Ahmadinejad comfortably ahead based on these data is to
assume that the people who did not express a preference divided
precisely the same as those who did answer the question. This
theoretical calculation produces a majority for the incumbent. The
question is whether such a calculation is justified - and the reality
is that even TFT did not make this leap in its pre-election analysis.
Rather it leaped in another direction, noting that "the race may
actually be closer than a first look at the numbers would indicate,"
because more than six in 10 respondents who expressed no opinion
"reflect individuals who favor political reform and change in the
current system." It went on to predict "that none of the candidates
will likely pass the 50 percent threshold."
Juan Cole makes a similar point and links to the original Terror Free Tomorrow report (pdf).
By Mark Blumenthal on June 15, 2009 10:17 AM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
My NationalJournal.com column for this week looks at lessons learned from last week's Democratic gubernatorial primary in Virginia.
Unfortunately, I had only three short paragraphs to devote to the "Google blast" purchased by the Deeds campaign. For more detail, I recommend the article I linked to by Nancy Scola of the Personal Democracy Forum. as well as this more in-depth account by ClickZ's Kate Kay that I found after filing the column. Kay interviewed Kyle Osterhout, partner at Media Strategies, the time buying agency that purchased television, radio and online advertising for the Deeds campaign and reported these details:
Deeds ran display and search ads targeted to people throughout Virginia for two months leading up to yesterday's primary. However, the last-minute Google ads were intended for Northern Virginian eyes only. Deeds was endorsed by The Washington Post, and the campaign believed that endorsement would carry more weight with Northerners, said Osterhout.
During the 28-hour Google blast, about 8.8 million ad impressions ran, and were clicked around 3,000 times. Like many display ad efforts, the click-through rate was tiny. But Get-Out-the-Vote campaigns like this one are not necessarily driven by online action: they're meant to get people to go out and vote.
In addition, they're often aimed at persuasion. "The goal was to push to primary voters that Deeds was the candidate that was endorsed The Washington Post," Osterhout told ClickZ News. Television ads mirrored that message. According to Osterhout, the campaign team plans to analyze ad performance according to Web site, ad size, and placement "to try to pinpoint what exactly worked the best."
The "Google Blast" advertising used by the Deeds campaign, as well as by Scott Murphy's campaign earlier this year in New York's 20th Congressional District, represent something of a threshold. Campaigns area starting to use Internet advertising as a means of persuading uncertain voters rather than as a prospecting tool to reach new donors or volunteers.
As the same time, Osterhout's comments and the click-through rates he provides tell us that the Deeds internet ads functioned as online yard signs. They were effective at communicating a message that could be boiled down to a single sentence. "The Washington Post endorsed only one Democrat: Creigh Deeds."
The estimate I cited for the reach of the Google ad network -- more than 80% -- comes from an analysis of ComScore data by Eric Frenchman, an online marketing and advertising consultant who worked on the McCain-Palin campaign and claims to have coined the term "Google Blast." A Google spokesperson told me that Google does not release official numbers on their network's reach, but she did point me to the most recent ComScore numbers like those that Frenchman used for his estimates.
By Mark Blumenthal on June 15, 2009 9:48 AM
| Permalink
| Comments (0)
| TrackBacks (0)
June 12, 2009
By Mark Blumenthal
Joshua Tucker suggests things to keep an eye on -- and points to pre-election polls -- in Iran.
Gallup finds women more likely to be Democrats, regardless of age.
Hotline On Call has some cautions on the Rasmussen Virginia poll.
Chris Cillizza asks whether the GOP faces better odds in NJ or VA.
Ed Kilgore calculates the regional distribution of the VA vote.
Nate Silver offers messaging advice to gay marriage advocates.
Andrew Gelman assesses state level estimates in support for gay marriage.
Ruy Teixeira breaks down the latest Gallup poll for TNRtv.
Alex Bratty explains a Rasmussen survey showing GOP gains on issues.
Gary Andres predicts government spending will be Obama's downfall.
Jay Cost looks at the politics of health care (more in Part II)
Jeff Merkley calls out Frank Luntz on health care from the Senate floor.
National Journal's Democratic insiders rate bipartisanship on health care less important than Republican insiders do.
Mark Mellman urges Middle East peacemakers to focus on moving and melding public opinion.
David Hill bemoans the decline of early benchmark polling by campaigns.
Steve Benen rounds up the questions "only Fox News would ask."
Brenden Nyhan shares research on correcting misperceptions that Obama is a Muslim.
Gordon Brown passes on Penn, hires Benenson.
Swing State Project teams with OpenCongress.org to create a 2010 Race Tracker wiki.
The Pew Internet Project reports on the social life of health information.
Kristen Soltis chats with political tech experts about social media and new technology.
Joel Rubinson reviews the ARF Online Research Council's findings on online data quality (via Korostoff).
Sysomos Inc. crunches Twitter data, finds 5% of users account for 75% of activity (via Rainie).
Zogby polls attitudes about the internet using an internet panel survey without a trace of irony (nor a disclaimer on potential response bias).
By Mark Blumenthal on June 12, 2009 5:43 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
Stuart Rothenberg's take on the lessons from the Virginia primary includes a slap at that odd last minute poll spin from the McAuliffe campaign (via Smith, link added):
[T]oo many press spokesmen caught up in their own spinning and campaigns often get too cute by half in trying to use poll numbers - that they often know are misleading - to energize supporters.
Chris Cillizza of the Washington Post quoted McAuliffe senior adviser Mo Elleithee on Election Day as saying, "In the last 48 hours, the lead that Senator Deeds had taken in the last week started to collapse," and an Elleithee Election Day get-out-the-vote e-mail cited the last night of a three-day tracking survey that showed Deeds and McAuliffe tied at 33 percent.
The e-mail was filled with disclaimers that the one-night results are "not definitive" and that the campaign never makes decisions "on one night's worth of interviews because the sample is too small." Nevertheless, the campaign released those numbers and constructed an argument based on them. The full three-night poll was never released, nor did Elleithee note that internal campaign polling over the previous week showed Deeds was pulling away and would win handily.
One McAuliffe campaign insider I spoke with after the results were in acknowledged that in the campaign's final days "we knew things looked really bad, but things looked volatile." That explanation is not convincing. In fact, those running the McAuliffe campaign knew very well what was happening.
The only conclusion possible is that the campaign was not telling the truth and that it was selectively using numbers that it knew should never be used to make a point that it knew was very dubious, at best.
A suggestion for journalists about this sort of last minute spin: Know the code. When a campaign trots out internal polling numbers that show a totally different trend from the public surveys, ask yourself, where is the pollster? If the people that produced the numbers are willing to bet their company's reputation on a counter-intuitive finding, take it seriously. On the other hand, if the pollster does not participate in the conference call or if their name appears nowhere on the memo, take it with a gigantic grain of salt. Better yet, take a walk.
To be clear: I have no knowledge, firsthand or otherwise, about what McAuliffe pollsters Pete Burnitz and Joel Benenson had to say about these numbers. For all I know they were in London on Tuesday pitching their newest client. Still, the absence of their names from that McAuliffe release is hard to ignore.
By Mark Blumenthal on June 12, 2009 3:41 PM
| Permalink
| Comments (0)
| TrackBacks (0)
June 11, 2009
By Mark Blumenthal
At least one unnamed Republican Senator continues to place a "hold" on the nomination of Robert Groves, and no one seems to know why. According to a report in the Wall Street Journal, earlier this week:
Dr. Groves, President Barack Obama's pick to lead the bureau, was approved easily by the Senate homeland-security committee in May, but Republicans blocked a confirmation vote last week. Senate Minority Leader Mitch McConnell of Kentucky said Republicans weren't yet in agreement on the nominee.
It is unclear why Republicans are blocking the vote. A McConnell spokeswoman, Jennifer Morris, said she had no information on the delay.
Yesterday, the New York Times editorial board took up the argument:
It is hard to imagine the public interest that is being served by the hold. It is easy, unfortunately, to imagine the political interest. A leaderless Census Bureau is unlikely to pull off an accurate count. Inaccurate tallies tend to favor Republicans, because a bad census misses hard-to-count groups that tilt Democratic, like minorities and immigrants, thus over-representing easy-to-count suburbanites who tilt Republican.
Kristen Soltis has written on our own site about the demographic challenge facing the Republican party in appealing to younger voters. Last year, she tells us, only 62% of voters under 30 were white. "Expanding the Republican Party's appeal to younger voters," she writes, "is inextricably linked with the issue of expanding the party's appeal to minority communities."
Separately, the recent Political Values study by the Pew Research Center American demonstrates continuing confidence in science, especially among the young. Only a third of Americans (34%), they report, agree "that science is going too far and is hurting society rather than helping it" (61% disagree). Not surprisingly, the report finds that younger Americans are the least likely to express discomfort with science. [Correction: I misread the report. Younger Americans are less likely to say that technology "s making life too complicated for me"].
I understand that the Groves nomination is about as obscure an issue as exists on the Congressional agenda and that partisanship trumps all else when legislators take up anything affecting reapportionment and redistricting. Still, someone needs to explain to me how the Republican Party is helping itself with this Luddite stand against a nominee with "bulletproof scientific credentials," especially when that stand projects hostility to the interests of minorities.
By Mark Blumenthal on June 11, 2009 2:16 PM
| Permalink
| Comments (0)
| TrackBacks (0)
June 10, 2009
By Mark Blumenthal
Regular readers will probably remember my that my father-in-law Frank Burstin, who passed away about a week before last fall's elections, was a survivor of the Auschwitz death camp. For that reason, as you may imagine, the news this afternoon about a shooting at Washington's Holocaust Museum hits pretty close to home for me and for my family.
But you don't know the half of it.
I have a special memory of Pop (as we knew him) from last summer. It was a few weeks before he received his cancer diagnosis, during what turned out to be his last visit to the Holocaust Museum. Because he lost his parents and all of his siblings to the Nazis, and because no grave site exists for any of his family, Pop made it a habit to visit the Museum at least once a year. It fulfilled for him the custom that many Jews practice of visiting the cemetery of loved ones once a year. I only got to accompany him on one of these visits, that one last year, along with my wife's nephew Jake.
I described him last year as "kind and optimistic soul," and he certainly was. But when he entered that museum, something changed. He was not unkind, but in that place, as I soon learned, he suffered no fools (nor anyone else).
We wandered into the museum, through the same doors and into the same foyer where shots rang out this afternoon. My wife had given us visitor passes that she receives as a member of the Museum. The lines were long, and it was not obvious which line we needed to stand in.
Pop was having none of it. He walked away from me and wandered up to the museum staffer standing at the head of the long line leading to the elevators that takes all visitors to the museum exhibits. I thought for a moment that Pop was going to ask directions. I was wrong.
He thrust out his arm in the direction of the staffer, displaying the number the Nazis tattooed on his arm at Auschwitz just a few inches from her face. Without making eye-contact and barely breaking stride, Pop kept walking. Understandably, the staffer barely blinked. She didn't make a move to stop him.
Pop kept walking right into the elevator that had just filled with the visitors that had been waiting in that long line. And even though the elevator was already quite crowded, he walked right in. Jake and I had to run past the guard to catch up. "Pop, Pop," I said, feeling a little embarrassed, hoping to talk him into at least waiting for the next elevator.
The staffer inside the elevator must have heard me, because he smiled, held the door and said with smile, "We have room for Pop. You guys too. C'mon in."
And up we went. I have been to the Holocaust Museum many times, but none as memorable as that visit.
About a month ago, in a conscious effort to carry on her father's tradition and to commemorate his birthday, my wife Helen paid her own solo visit to the Museum. She arrived at the end of a busy work day, in a rush, just a few minutes before closing time. Unfortunately, given the late hour, they had run out of the candles usually provided in the Hall of Remembrance for visitors to light and leave in the niches of the outer walls.
Already feeling emotional -- her dad had passed away just six months before -- she broke down sobbing.
A staffer nearby immediately came to her assistance, asking if she needed help. She explained, and the gentleman asked her to wait. He soon returned with a candle, explaining with a conspiratorial wink that he kept his own special supply for such emergencies.
The guards and staff at the Holocaust Museum have a special duty. The do more than just protect and operate one of Washington's many heavily trafficked museums. On a daily basis, they help open the doors to the elderly survivors of the atrocities of World War II. As my stories attest, they do it with a remarkable degree of kindness and professionalism.
As far as I know, the Holocaust Museum personnel that we encountered were not armed guards, though it is possible they were. But when I heard about the shooting this afternoon, and more specifically that at least one of the victims is a security guard now apparently in critical condition, it struck very close to home.
This is personal.
As far as I am concerned, the staff members of the Holocaust Museum are part of our family and the Museum itself is hallowed ground. We pray for the recovery of the wounded guard. "Never take your guard force and security people for granted," William Parsons, the museum's chief of staff said on television a few minutes ago. Our family never will.
A very sad update: MSNBC just reported that the guard, Officer Steven Tyrone Johns, has passed away. We are all mourners tonight.
Update 2 - Spurred on by the commentary of my colleagues Ta-Nehisi Coates and Jeffrey Goldberg, I want to add one piece of relevant information. Officer Johns and all of the staff I described above were African-American or (in one case) African.
By Mark Blumenthal on June 10, 2009 3:48 PM
| Permalink
| Comments (14)
| TrackBacks (0)
By Mark Blumenthal
What a difference perception makes. Last year, the New Hampshire Democratic primary produced an unprecedented polling "fiasco," also described as "one of the most significant miscues in modern polling history" (to quote two of the most respected voices in political polling). This morning, I see no such angst, and for good reason. Creigh Deeds won Virginia's Democratic primary for governor by a crushing margin after the final polls had shown him leading by a double digits and trending sharply upward.
But look closer. If we simply compare the final polls to the actual results, the "polling errors" were actually bigger in Virginia last night than in New Hampshire. In New Hampshire, as the table below shows, the final polls as summarized by our trend estimates of the vote for Barack Obama and John Edwards came remarkably close to their actual percentages but understated Hillary Clinton's support by nearly 9 percentage points. Last night, our trend estimates came within tenths of a percentage point from the actual votes won by second place finishers Terry McAuliffe and Brian Moran, but they understated Deeds' final tally by a whopping 13 percentage points.
The crucial difference, of course, is that the Virginia polls gave us clear direction of both the winner and the final trend, while the New Hampshire polls pointed us in the wrong direction on both. So while we may have been a bit surprised by the margin last night, we had ample warning that uncertain voters were "breaking" to Deeds over the final days of the campaign.
Still, it is worth noting that simply extending our trend lines on either Deeds' support or his margin over McAuliffe from Sunday through Tuesday does not explain or predict the ultimate margin. Here, courtesy of Charles Franklin, is a chart that extends our trend estimate for Deeds' support using either our standard estimate (the solid blue line), the more sensitive estimate (the dashed blue line), or a straight line ("linear fit") based only on the polls conducted since completed after May 15. As Charles writes via email, "All three are essentially the same as of election day, at 39% or so."
You see the same pattern if we plot Deeds' margin over McAuliffe (the Deeds percentage on the poll minus the McAuliffe percentage). It shows the same sharp upward trend with Deeds clearly ahead but, again, not by as much as his actual 23 point margin.
My point here is not to bash the polls in Virginia. To the contrary, the much derided automated surveys conducted by Public Policy Polling (PPP) and SurveyUSA, as well as the live interviewer polls from Research 2000 and Suffolk University, provided a consistent, clear and apparently accurate picture of the trend in voter preferences (though they were more divergent about the level of candidate support seen earlier on). As with Clinton in New Hampshire last year, we can probably never know for certain whether their final estimates understated Deeds support or whether he benefited from a virtually monolithic "break" of undecideds in his direction in the closing hours.
The point, which I tried to make over the weekend, is that late shifts and polling errors are a lot more common in primary elections because turnouts are smaller and likely voters are harder to select, because partisans are not locked into choices based on party affiliation (as they are in general elections) and because the dynamics of contests featuring three or more viable candidates produce more volatility on voter preferences.
On a slightly different topic: I wrote earlier this week about the potential benefits of sampling low turnout primaries with lists of registered voters, a subject of sometimes fierce debate among pollsters. I am not sure we can draw firm conclusions on that subject from the final wave of polls in Virginia, since polls using both list samples (PPP and Suffolk University) and those using random-digit-dial samples (SurveyUSA and Research 2000) obtained generally similar results and tracked similar trends over the last week. Earlier on, however, the list sample polls tended to show lower support for McAuliffe, although again we are limited by our small "sample size" of pollsters and inability to control for other issues (such as question format and degree of screening).
That said, the survey world needs to take more seriously the argument that PPP's Tom Jensen made last night. This Virginia primary, he wrote, "was the perfect race to be polling using the voter list and automated calls." In particular, I wish someone in would devise a a randomized controlled experiment (and place the results into the public domain) to test Jensen's implicit assertion about non-response bias and automated surveys:
When you're dealing with an automated poll folks who don't intend to vote don't feel the sort of social pressure they might feel from a live interviewer to participate. So folks who didn't plan to vote didn't bother to answer the poll. No harm done. You don't want a high response rate from people who aren't going to vote.
In other words, if we hold all other factors constant -- something never possible with after-the-fact comparisons of results from different pollsters -- does calling with an automated method do a better job selecting truly likely voters than calling with live interviewers? That could be done with an experiment that samples with a registered voter list and updating after the election to validate turnout.
Let's give Jensen some due credit. Using his survey measurements he predicted a turnout "somewhere in the range of 300,000 voters," while others were less certain or predicted lower numbers (though apparently not the McAuliffe campaign). With 99.8% of precincts counted, over 320,369 votes were cast last night. Even Nate Silver has trouble predicting turnout.
By Mark Blumenthal on June 10, 2009 8:44 AM
| Permalink
| Comments (2)
| TrackBacks (0)
June 9, 2009
By Mark Blumenthal
10:20 p.m. - A nearly complete count with all but five precincts included: Deeds 49.7%, McAuliffe 26.4%, Moran 23.8% with ver 319,000 votes cast.
9:23 p.m. I am going to leave the deeper "lessons" until tomorrow, but with 97.3% of the precincts counted, Deeds leads McAuliffe by a massive 22-point margin -- Deeds 49%, McAuliffe 27%, Moran 24% and 305,866 votes counted so far. PPP's Tom Jensen was right when he guessed the turnout would be "around 300,000." Even the final polls understated Deeds ultimate margin, although they clearly had the trend moving in the right direction. More tomorrow.
8:47 p.m. - Here are the first four heavily (presumed) African-American precincts to report in Richmond City (Obama '08 GE percentage in parentheses):
- Pct. 301 (96%): 45% McAuliffe, 30% Deeds, 25% Moran
- Pct. 303 (98%): 45% McAuliffe, 28% Deeds, 27% Moran
- Pct. 304 (96%): 49% McAuliffe, 28% Deeds, 23% Moran
- Pct. 305 (96%): 40% McAuliffe, 31% Deeds, 29% Moran
8:20 p.m. Meant to include this from 538's Ed Kilgore, based on this CD map:
With the votes now pouring in, Creigh Deeds is winning in nine of
Virginia's eleven congressional districts (Moran narrowly leads in the
8th on the strength of his Alexandria performance, but the lead
probably won't hold, and McAuliffe leads narrowly in the 3d, with
Richmond still out)
8:17 p.m. I'm guessing this comes as no surprise to anyone reading this, but Nate Silver's FiveThirtyEight is also live blogging tonight.
8:12 p.m. Here are are few more presumed-to-be heavily African American precincts in Newport News:
- Pct. 404 (98% Obama): 52% McAuliffe, 26% Deeds, 22% Moran
- Pct. 414 (99% Obama): 46% McAuliffe, 32% Deeds, 23% Moran
So far, none of the many heavily African American precincts in Richmond City have reported, which probably means that Deeds statewide margin will narrow a bit from where it is now.
8:07 p.m. - So did the African American vote break decisively to Terry McAuliffe (discussed in more detail here)? Not nearly enough to offset Deeds margins elsewhere apparently.
I'm
going to assume that any precinct that Barack Obama
carried with more than 90% of the vote in 2008 is virtually all African
American. Here are a handful of precincts reporting in Newport News that are 90%+ black:
- Pct. 303 (98% Obama): 47% McAuliffe, 34% Deeds, 19% Moran
- Pct. 402 (99% Obama): 52% McAuliffe, 25% Deeds, 24% Moran
- Pct. 405 (98% Obama): 40% McAuliffe, 37% Deeds, 23% Moran
8:00 p.m. Sounds like AP just projected Deeds the winner - local radio station WTOP just called it for Deeds.
7:50 p.m. A quick check shows Deeds winning nearly 50% in precincts counted so far in Arlington City and Fairfax County and roughly a third of the vote in Moran's true home base in Alexandria City. If that holds up, this will not be close.
7:45 p.m. This post is likely to be more of an occasional election night update than a live-blog. I was battling another torrential rain-storm here in DC and just got home. The polls closed in VA 45 minutes ago and results are starting to come in. It doesn't sound like the Associated Press has declared a winner in the Governor's yet, although they just declared a winner (Jody Wagner) in the Lt. Governor's race.
By Mark Blumenthal on June 9, 2009 7:45 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
Tom Jensen takes a stab at predicting the Virginia outcome at specific turnout levels.
McAuliffe's campaign says their Monday night calling shows a close race.
Tom Schaller sees Virginia as test case for the "Martin Effect."
The Post finds support for Deeds among a dozen voters exiting a Northern Virginia polling place.
Steve Singiser considers the case for the three candidates in a low turnout (and shares anecdotal reports on this morning's voting).
Josh Putnam examines historical patterns in New Jersey governor's race polling.
Ken Strasma talks to FiveThirtyEight about his microtargeting work for the Obama campaign.
Glen Bolger urges Republicans to be patient about perceptions of Obama.
The Pew Internet Project interviews local elected officials about their Internet usage.
Bill McInturff and Al Quinlan share results from their survey on health care provention (sponsored by the health advocacy group, Trust for America's Health).
Bernard Avishai reports details of more Israeli polling (via Marshall)
Anthony Wells assesses UK pollster performance and accuracy in the recent Euro elections.
And the placement of this display ad indicates quite a Google-Bomb (yes, seen only from browsers in the U.S, but still...).
By Mark Blumenthal on June 9, 2009 3:33 PM
| Permalink
| Comments (1)
| TrackBacks (0)
By Mark Blumenthal
One of our alert readers noted two paragraphs in yesterday's New York Times report on today's Democratic primary for Governor in Virginia. First this:
A close ally of Bill and Hillary Rodham Clinton, Mr. McAuliffe, 52, is a multimillionaire businessman with an outsize personality who held an early lead in statewide polls based largely on his ability to out-talk and outspend his opponents.
Then later in the story, this:
After most polls had the three candidates in a statistical dead heat for most of the race, Mr. Deeds opened up a slight lead over the weekend, but many voters were still undecided and turnout was expected to be low.
Never mind that the characterization of McAuliffe's lead in "statewide polls" in the first paragraph clashes with the supposed "statistical dead heat" shown by "most polls . . . for most of the race" (the first paragraph had it right). Which polls is the Times talking about? After all the Times' official Polling Standards forbid the publication of of "interactive voice response polls," surveys "conducted by Democratic or Republican pollsters or privately-sponsored organizations or interest groups," and labels as "questionable" those surveys based on registered voter lists.
Our compilation includes seventeen publicly released, statewide surveys in Virginia that reported a vote preference question on the Democratic primary race. Of the seventeen, eleven were automated (IVR) polls (from PPP and SurveyUSA), two were conducted by Moran's Democratic pollster (Greenberg, Quinlan, Rosner Research) and one more (from Suffolk University) was conducted using live interviewers but drew a sample from the list of registered voters. That leaves three polls conducted using live interviewers and a random digit dial sample by Research 2000 on behalf of the DailyKos website. You can decide whether DailyKos counts as a "privately-sponsored organization or interest group."
More to the point, the only surveys that showed Deeds opening up a lead "over the weekend" were the automated/IVR polls done by PPP and SurveyUSA.
Similarly, the Washington Post tells us this morning "surveys" have been showing "remarkable volatility in the race's final days," but since those surveys do not meet the Posts standards for publication, they will not report on them.
So apparently the recent findings from these automated polls are newsworthy, but who did them, what they show (specifically) and why are not. Got it?
By Mark Blumenthal on June 9, 2009 12:18 PM
| Permalink
| Comments (0)
| TrackBacks (0)
June 8, 2009
By Mark Blumenthal
Some notes about the two final surveys out in Virginia over the last 24 hours from automated pollsters PPP and SurveyUSA. Both show Creigh Deeds surging ahead 40% and 42% of the vote, respectively. PPP shows a big move from a previously big undecided to Deeds; SurveyUSA shows a fall-off in support for Terry McAuliffe. And if you want indirect evidence that the live-interviewer, internal campaign polls are showing the same thing, just watch the what the McAuliffe and Moran campaigns were doing over the weekend: Attacking Deeds.
Deeds appears to be on track for a victory, as he is doing well on two of the three keys to the outcome I reviewed on Saturday:
First, his paid media gamble in Northern Virginia appears to be paying off. SurveyUSA shows him surging ahead to a 12-point lead over Moran in their "Northeast" Virginia region (42% to 30%), PPP also shows Deeds ahead of Moran in the 703 area code but by a not-quite statistically significant three points (38% to 35%).
Second, on these two snapshots at least, McAuliffe's support from African-Americans continues to fall short of the decisive that might tip the balance in a closer race: Both PPP and SurveyUSA show McAuliffe a few points head of Deeds (McAulliffe up 32% to 31% on the PPP poll, 38% to 32% according to SurveyUSA).
Nonetheless, I stand by my warning over the weekend about the uncertainty of these estimates: All evidence points to a continuing Deeds surge, but I would not wager much on the precise margin.
One more thing: Yesterday, the Moran campaign posted "results" from their voter ID phone-banking showing a "surge" in support for their candidate (emphasis added):
Yesterday, the Moran campaign received some of the best results of the campaign and saw a notable uptick in support while phone banking likely primary voters. The campaign made 36,478 calls to highly likely primary voters in Northern Virginia and saw Moran's consistent lead there surge after information about the records of his two opponents was made clear. In Hampton Roads, Moran's canvass calls reached 23,454 likely primary voters and found that Moran's support has increased significantly, moving the race into a virtual three-way tie.
In Richmond, Moran's canvass reached 22,399 likely voters and found that Moran is now within the margin of one of his opponents, while leading the other by several percentage points. And in Roanoke, where Moran just last week began his advertising campaign, his support has risen 150 percent among likely voters.
Let's start with the phrase in boldface, which makes clear that the measurements cited came after the callers presented negative information about Deeds and McAuliffe. At a campaign's 11th hour, a vote question following negative "message testing" (and that's charitable) is not exactly a fair way to measure the current standings. Most casual readers will probably want to stop there, but I'm getting ahead of the story.
It is tempting to write this release off as a bit of last minute spin by a campaign desperate to deliver some good news to it supporters. But in case some are confused, let's be clear: A telephone canvass used to identify supporters is not a poll. It does not involve a random sample of voters and depends on volunteer callers.
Campaigns generally take two approaches to voter ID calls: Either they make "blind" calls that impersonate a poll (the callers say they are taking a "survey" and do not identify the sponsor) or they identify themselves honestly (which means, in this case, saying they are calling on behalf of the Moran campaign).
If the callers in this case identified themselves as Moran supporters -- and if they had not been reading negative information aloud about Deeds and McAuliffe -- the results of a canvass might be useful for identifying Moran's supporters (the ostensible purpose of such a canvass) but not as useful for identifying supporters of other candidates. These voters are more likely to hang up or feign indecision to avoid the "social discomfort" that comes with telling the volunteer caller they are going to vote for someone else (oh, if only I had a hundred dollars for every time a client's voter ID calling looked better than my tracking poll. And even then, the useful "data" would come from a vote preference question asked before trashing Deeds and McAuliffe.
If on the other hand the callers failed to identify themselves as calling on behalf of the Moran campaign, then this release amounts to bragging about "results" from a straight-up push poll: calls made under the guise of a survey that intend to communicate a message to as many voters as possible, in this case a negative message. That is my definition of true push polling and the definition of the American Association for Public Opinion Research (AAPOR).
So, Moran campaign, which is it?
The Moran campaign did not immediately respond to a voice mail asking for comment.
P.S.: In a related item, Jerome Armstrong links to both the PPP survey and the Moran canvas numbers and argues that the PPP numbers are off because they expect a 300K turnout, a "radically different model" that relies on "a lot of independents and Republicans voting in the Democratic primary." He cites the Moran canvass numbers as evidence that a smaller turnout bodes better for Moran, then concludes:
A 14-16% [Deeds] lead seems unbeatable, but remember that PPP is predicting a huge turnout too. I know both McAuliffe and Moran have much better field organizations than Deeds, by far-- its not even close. Deeds has gotten all the breaks in the polls; now, does he have the votes? The turnout will tell.
The problem is, as Tom Jensen notes in a blog post this morning, PPP's survey includes a cross-tabulation that essentially models a much lower turnout scenario showing Deeds leading by an even larger margin:
Among voters who turned out for one of the low intensity contests between 2005 and 2007 Deeds is at 46% with Moran at 26% and McAuliffe at just 19%. Among frequent primary voters 48% have an unfavorable opinion of McAuliffe with just 33% viewing him positively.
Among what we would describe as more casual primary voters- those who only turned out for Clinton/Obama- Deeds has a much narrower lead with 36% to 30% for McAuliffe and 22% for Moran.
[Prior association disclosed: David Petts, pollster for the Deeds campaign, was my business partner though 2006].
By Mark Blumenthal on June 8, 2009 5:22 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
My NationalJournal.com column for this week is now posted. It picks up where last weeks' left off, with more thoughts on what media pollsters can do to get better readings early on low turnout, off-year primaries.
By Mark Blumenthal on June 8, 2009 12:01 PM
| Permalink
| Comments (2)
| TrackBacks (0)
June 7, 2009
By Mark Blumenthal
PPP teases partial results from their VA poll again.
The Washington Post publishes the Suffolk University and Kos/Research 2000 polls.
Nate Silver sees parallels between Virginia 2009 and Iowa 2004.
Gary Langer evaluates how words affect poll questions on affirmative action.
Kos discovers that only 8% believe Sotomajor is a racist.
Chris Bowers assesses how unemployment could affect Obama's approval rating.
Jo-Ann Mort shares results of a poll of Israeli's showing support for settlement evacuation (Marshall comments, Mort responds)
Andrew Martin and Kevin Quinn defend their Supreme Court justice ideology estimates.
Drew Conway asks some questions about that new statistical research on Twitter.
Flowing Data reviews the rise of the data scientist.
Dana Stanley posts interviews from the CASRO conference.
Chris Anderson will live blog Apple's WWDC Keynote.
Pew Research Center finds 97% of teens play video games.
By Mark Blumenthal on June 7, 2009 11:23 AM
| Permalink
| Comments (0)
| TrackBacks (0)
June 6, 2009
By Mark Blumenthal
Politico's Andy Barr and Josh Kraushaar penned a nice summary yesterday on the widespread skepticism about the pre-election polls in Virginia. It quotes yours truly as saying it is "virtually impossible" (my words) to "accurately poll" (their words) this race. That may sound a bit strong. It is certainly possible to conduct a survey on the race, and some or all of the polls may ultimately provide an accurate result. But they may also miss by a mile, both because of the challenge of identifying likely voters and because of the chance of last minute shifts. What is "virtually impossible" right now is high confidence that the current polling results will predict who will win or their margin of victory on Tuesday.
Remember the New Hampshire primary last year? Remember how well the polls did there? Virginia shows similar signs of volatility. The Virginia polls show a consistent late trend favoring one candidate with another dropping (ditto in NH in 2008). The four pollsters that have tracked the race show Creigh Deeds gaining and Terry McAuliffe dropping. One poll shows these shifts occurring despite all three candidates maintaining relatively high favorable ratings (ditto in NH). And all of the surveys indicate considerable uncertainty. Our chart shows a multi-poll trend estimate of roughly 18% that are still completely undecided (as of this writing). Three pollsters also asked voters this week if they might still change their minds and found anywhere from 52% to 60% still either totally undecided or uncertain about their current preference (uncertainy on a similar question on the UNH survey was 23% just before the NH primary).
Late shifts and polling errors are a lot more common in primaries than in general elections, partly because it is harder to define the likely electorate and partly because fewer voters are locked into a choice based on party affiliation. In this race, I see at least three factors -- call them keys to the election -- that could well produce a different result (in the leader or in the margin of victory) than what polls currently suggest.
1) Turnout -- I wrote about the turnout puzzle previously, and my post on Wednesday noted that surveys are consistent in showing that a bigger turnout tends to work in Terry McAuliffe's favor (although that pattern could change over the final week).
The really critical question, however, is not so much the level of turnout but (as one helpful reader put it via email) "the dispersion of the turnout based on region." How much of the vote will come from Northern Virginia (where Brian Moran is strong), how much from the rural counties (Deed's base) and how much from the other urban centers in Virginia? Put another way, whose supporters are most energized and ready to vote?
The dispersion question may be most relevant for Northern Virginia, Brian Moran's base. Analysts define the Northern Virginia region differently, but by one definition it surged from about a third of voters in the 2005 Democratic primary to nearly 45% in 2006. Most expect rural interest in Creigh Deeds to boost turnout downstate, but we really won't know the answer until the votes are counted.
2) African-Americans -- If you believe the composition reported by the various public pollsters, African Americans will constitute roughly 30% of the vote on Tuesday. Their preferences remain a wild card.
Although none of the candidates began with a pre-existing base of support among African Americans, Terry McAuliffe is counting on a big vote there because of his long time association with President Bill Clinton. This week's Suffolk University poll gave Clinton a 79% favorable rating among African-American primary voters. McAuliffe has campaigned with Clinton, has run radio ads featuring the former president and just this week sent out direct mail to black voters this week touting support from Virginia Congressman Bobby Scott and the endorsement of the Richmond Free Press, an African American newspaper.
Polls have shown inconsistent results among African-Americans. SurveyUSA has consistently shown McAuliffe running well ahead (38% on their most recent survey) with a relatively small undecided (11%). The most recent surveys by PPP, Research2000 and Suffolk University have shown a close, three-way race with undecideds over 20%. PPP has shown an erosion of support for McAuliffe among African Americans (from 37% in early May to 24% last week), but a consistently huge undecided number (38% in early May, 36% last week). In their most recent poll, PPP found that 60% of African Americans were either completely undecided or could still change their minds (see our chart for links to all polls).
Some speculate that African American turnout will be relatively low, and the greater uncertainty does suggest a lack of enthusiasm. Yet the most extreme speculation of a low black turnout still puts the American-American composition at at least 20%, probably closer to 25%. And the results above suggest that when pushed, African Americans voters tend to break to McAuliffe. A big break in that direction -- especially if he can run his Black support up to 60% or more -- could give McAuliffe with the margin he needs to win.
3) Northern Virginia - Voter preference in the areas of Northern Virginia that fall in the Washington DC media market are probably the biggest unknown in this final weekend, both in terms of turnout and voter preferences. Given the cost of broadcast television and Brian Moran's base of support in the region, all three campaigns have directed their paid communication elsewhere until last week. As such, Northern Virginia voters are engaging in the race later with a a greater potential for last minute shifts.
Brian Moran represents a Northern Virginia district in the legislature, and his better known brother represents the area in the U.S. Congress. So he has consistently led in the region with support in the high 30s or low 40s, depending on the poll, with undecideds typically lower in Northern Virginia than other regions. Although Moran has run television advertising in other markets, he is gambling that a combination of an extensive field campaign and direct mail can motivate a large turnout from his base.
Deeds has typically polled a distant third in Northern Virginia. He was still running third -- with 16% to 23% support-- on the polls released this past week. However, he received a huge potential boost in a surprise endorsement by the Washington Post in late May, an endorsement that has often proved decisive in Democratic primaries in DC and suburban Maryland. Believing they have an especially potent message, the Deeds campaign dug deep to fund a television ad touting the endorsement. It started airing Wednesday night on both broadcast and cable television, and I am told the broadcast component is in the neighborhood of a light 500 gross ratings points.
McAuliffe is doing in Northern Virginia what he has been doing throughout the state: Use his big fundraising advantage to try to overwhelm the other candidates with all forms of paid communication. He has been running television advertising in the DC market since last weekend and, from my own viewing of local stations, appears to be airing more ads than Deeds.
So in Northern Virginia you have a variety of wild-cards: One candidate with a base wagering on direct mail and field organizing, two others with less support dominating television and the still uncertain impact of the Washington Post endorsement, all of it in a region where a lot of voters just started paying attention.
***
We believe our Virginia chart provides as good an indicator as any of recent trends. It shows that Deeds has been gaining and McAuliffe falling. I would not bet much, however, on the accuracy of the levels of support for each candidate as reported by these surveys. Variations in the dispersion of turnout could produce a significantly different result, and dramatic late shifts -- especially among African Americans or in Northern Virginia -- are still a distinct possibility.
By Mark Blumenthal on June 6, 2009 12:17 PM
| Permalink
| Comments (1)
| TrackBacks (0)
June 5, 2009
By Mark Blumenthal
ABC's Gary Langer reports that a mystery senator has put a "hold" on the nomination of Robert Groves:
Three weeks after Robert M. Groves sailed through his confirmation hearing to lead the U.S. Census Bureau, a Republican U.S. senator has placed the nomination on hold.
As for who and why, it's Mystery Hour at the U.S. Senate.
A Democratic staffer this afternoon suggested to our senior congressional correspondent Jonathan Karl that signs seemed to point to Sen. Robert Bennett, R-Utah. Nope: "It's not us," says Bennett's spokesman, Tara Hendershott. "We don't have a hold."
Someone clearly does. The majority leader, Sen. Harry Reid, D-Nev., asked this evening for unanimous consent to move the Groves nomination to a vote - and the minority leader, Sen. Mitch McConnell, R-Ky., objected.
For those who have not been following the story, Groves is a hugely respected figure within the survey research community (I interviewed him at the 2008 AAPOR conference).
If any of Pollster's very well informed readers has any tips as to which senator is behind the hold, please drop me a line.
Update - Congress Daily's Carrie Dann has more, including this:
One Democratic aide said the apparent hold on Groves' nomination is
not cause for concern, considering how quickly the confirmation vote
was brought to the Senate floor.
"Sudden movements make people nervous," said the aide. "Next week we'll know if there's a real problem or not."
By Mark Blumenthal on June 5, 2009 2:33 PM
| Permalink
| Comments (0)
| TrackBacks (0)
June 3, 2009
By Mark Blumenthal
Apologies for missing this, but on Monday Patrick Murray, founding director of the Monmouth University Polling Institute, posted a terrific primer on this Tuesday's New Jersey primary. His post includes an intriguing description of their methodology and how they "modeled" turnout that may have lessons for the polls out now on next week's Virginia primary:
The Monmouth University/Gannett New Jersey Poll released two weeks ago showed Christie with an 18 point lead - 50% to 32% for Lonegan. For the record, that poll was conducted using a listed sample of registered Republican voters in the state who were known to have voted in recent primaries. It was further screened to determine the propensity of voting in this particular election (based on a combination of known past voting frequency and self-professed likelihood to vote this year). In the end, our model assumed a turnout of about 300,000 GOP voters on June 2 (give or take 10,000).
[...]
Variations in turnout tend to have more impact on primary results than they do on general elections. In general elections, the preferences of non-voters tend to line up fairly well with those who actually go out to the polls on election day. However, for primary elections, particularly with an ideologically-fractured GOP electorate, a factor of just a few thousand voters simply deciding whether or not to show up can swing a close race.
It doesn't look like we have a particularly tight race in this case, although that 18 point poll gap may have narrowed since our last sounding on May 20. I did re-examine our data using alternative turnout estimates. If the GOP primary turnout model is set to well above 430,000 - i.e. a 40-year record turnout for a non-presidential race - the Christie margin in our poll grows to 23 points. Alternatively, if the turnout model is pushed down to about 200,000 - i.e. a typical U.S. Senate race - the gap shrinks to 13 points. That's a swing of 10 points based on turnout alone!
I asked Murray if he would provide us with some post-primary thoughts via a "guest pollster" post, and if all goes well, we should have that posted for you tomorrow. But consider his observations about turnout in New Jersey in the context of the polls released in the last week or two in Virginia:
- The two pollsters that have shown Terry McAulliffe doing best -- SurveyUSA and Research2000 -- have used random digit dial (RDD) samples that cannot use the sort of actual vote history information available for individual respondents on list samples.
- The two pollsters that have sampled using registered voters lists -- Public Policy Polling (PPP) and Moran's pollster, Greenberg-Quinlan-Rosner (GQR) -- have consistently shown McAuliffe running 7 to 10 percentage points lower than the polls using RDD samples. I reported details of the PPP sampling method here. I assume that GQR uses lists in this race because their first release says they identified likely voters using both "vote history in Virginia and self-reported likelihood to vote in the upcoming gubernatorial primary" (emphasis added). Again, vote history is only available with a voter list.
[Note: If you click on any data point in our Virginia chart, embedded below, you can connect-the-dots for surveys from individual pollsters and see how each compares to the overall trend]
- On their last two polls, PPP provides crosstabulations that compares two groups: (1) households with vote history in either of the very low turnout primaries in 2005 or 2006 with (2) households where voters participated in only the much higher turnout 2008 presidential primary. Both polls show Deeds doing better (by 8-10 points) in the lower turnout households. McAulliffe scored 9 points lower in the low turnout households two weeks ago, but just two points lower earlier this week.
- SurveyUSA's summary of their latest survey out today includes these findings that suggest a similar correlation: McAuliffe does best among the subgroups with the historically lowest levels of turnout:
McAuliffe's constituents are Independent and young. In SurveyUSA's turnout model, 20% of likely Primary voters are Independent. If this group votes in smaller numbers, McAuliffe's support is overstated here. In SurveyUSA's turnout model, 19% of likely voters are age 18 to 34. If this group votes in smaller numbers, McAuliffe's support is overstated here.
Combine these findings with the considerable self-reported uncertainty -- half (52%) of SurveyUSA's respondents and 44% of the voters on the last PPP survey say could still change their minds -- and we get a race where the final result may look very different from whatever the final round of polls "predict." Hang on to your hats.
PS: Several big unknowns remain in this race, but one big one is now a bit clearer. Creigh Deed's campaign just sent out a release announcing that they will begin airing a television advertisement touting his recent Washington Post endorsement "on broadcast and cable stations in Northern Virginia." Note, however, that the release provides no details about how much time Deeds is buying on the very expensive DC broadcast stations (that also reach into Virginia, Maryland and DC). If they are committing to a decent sized broadcast buy in the DC market, it's a major gamble. If any of our readers catches this new ad on Washington DC broadcast television, please email me or leave a comment below (or email me).
[Prior association disclosed: David Petts, currently the pollster for the Deeds campaign, was my business partner though 2006].
By Mark Blumenthal on June 3, 2009 10:31 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
Gallup reports results on how Arab Countries view the U.S. and how Americans view Muslim nations (plus video).
Gary Langer examines the Gallup data, sees views of the U.S. among muslims as better but far from good.
Mark Lynch ponders the meaning and limits of surveys of the Arab world (via Sullivan).
Liz Sidoti summarizes the AP-GfK poll results on torture and closing the Guantanamo Bay prison.
Chris Cillizza walks back a reference to a PPP poll; PPP's Tom Jensen notes no such walk-back on a reference to the Moran campaign's internal poll.
The Pew Project for Excellence in Journalism finds the Sotomayor nomination topping the news agenda.
Chris Bowers and Andrew Gelman go back and forth on whether Mike Dukakis would have won if turnout in 1988 had been like 2008 -- more from Bowers here.
Gary Jacobson reviews (pdf) the 2008 elections; Chris Good summarizes.
David Hill believes Michiganders are "resolute, selectively optimistic" and "willing to wait for a genuine long-term solution."
Mark Mellman reviews psychological research on the role of personal background in decisionmaking in the context of the Sotomayor nomination.
Alex Lundry posts a data visualization of numeric ideological estimates for Supreme Court Justices; Andrew Gelman has doubts about the numbers.
Bluegrass Politics reports on two Kentucky campaigns sparring over released internal polling; Tom Jensen weighs in.
Nate Silver finds lower abortion rates in states with more self-identified "pro-life" adults.
Chris Bowers says the culture wars will always be with us.
Chris Weigant charts Obama's approval in comparison to previous presidents.
Jim Snyder reports on the use of microtargeting by lobbyists and issue advocacy groups.
Andrew Gelman agrees with Simon Jackman on Internet polls.
DCAAPOR will host a discussion of the "current state of telephony in the US" on June 15.
Carl Bialik reports on the creation of a new, experimental government agency in the U.K. with authority to oversee, monitor and audit all government produced statistics (more on his blog).
NDN sponsors new polls conducted by Pete Brodnitz and Celinda Lake.
Joab Jackson considers the challenges facing data.gov (via Lundry).
Bill Heil and Mikolaj Piskorski research Twitter usage; Drew Conway asks some critical questions.
By Mark Blumenthal on June 3, 2009 2:15 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
One reader emailed to take strong exception to my use of the word "misleading" in the following sentence from my column posted yesterday on NationalJournal.com:
One reason for the misleading early numbers in 2006 may have been that Quinnipiac sampled all self-identified registered Democrats rather than a narrower subset of likely primary voters. Their May 2006 sample of 528 Democrats, for example, amounted to 34 percent of the full sample of 1,536 registered voters they interviewed. Yet the actual Democratic primary turnout amounted to just 15 percent of Connecticut's active registered voters.
I will grant that I could have chosen a less loaded word than "misleading," as some will hear it as an insinuation about the pollster's motives or the accuracy of data. For the record, I do not believe that anyone involved in producing the Quinninpiac Poll meant to mislead anyone, and did not mean to imply that the data they reported were inaccurate. The record should show that once they shifted to reporting vote preferences among a narrower group of "likely voters," they showed Lamont running much closer to Lieberman in June, "inching ahead" in July and ultimately leading by a wide margin in early August. Their final poll showed Lamont leading by six percentage points. He won by four -- that's as close as any poll should expect to get.
The larger point I was trying to make with the column is that we mislead ourselves -- and by we I mean all of us, pollsters, journalists, campaigns, political junkies -- whenever we treat samples of a third to half of adults in a state as a meaningful measure of the preferences of "likely primary voters" when the actual turnout is typically a much smaller fraction of adults.
My use of the Quinnipiac Poll was also largely a coincidence. They happened to produce two polls last week with primary head-to-head questions, one in Connecticut, and a similarly designed poll in Connecticut three years ago. However, their practices in terms of sampling primary voters are very similar to those used by most other media pollsters.
By Mark Blumenthal on June 3, 2009 11:40 AM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
Kristen Soltis, a regular contributor here at Pollster.com, summarized her views on how the Republican Party can win back younger voters for the Huffington Post. Her bottom line:
In order to begin that effort, the GOP needs to have a positive message and vision that focuses on outcomes that matter to young voters. Right now, a lot of what Republicans are talking about is "less taxes" and "smaller government." But young voters are less convinced than older generations that the government tends to be inefficient and wasteful.
Among other issues, she also confronts the lack of diversity that was the subject of a widely read summary from Gallup this week that showing that 89% of Republican identifiers are white (or more specitically, non-Hispanic white) and 63% are white conservatives. Soltis:
Longer term, the Republican Party has to confront the issue of diversity. If the Republican Party retains a brand as the party tailor-made for conservative older white males, it will not survive for long. Consider the fact that younger voters represent a more ethnically diverse cohort than other generations. The issue of winning the youth vote is more and more inextricably linked to winning support among Hispanics and African-Americans.
There's much more, and it's worth clicking through for a full read.
By Mark Blumenthal on June 3, 2009 11:03 AM
| Permalink
| Comments (1)
| TrackBacks (0)
June 2, 2009
By Mark Blumenthal
"Sometimes the magic works," said Chief Daniel George in the 1970 classic flim Little Big Man, "and sometimes it doesn't." The same can be said about the loess regression trend lines we plot in our charts.
When we plot pre-election poll results from various pollsters on the same charts, the trend lines usually have the helpful characteristic of minimizing the impact of outlier results and pollsters with consistent "house effects" on the overall estimate. In other words, if one of five or ten pollsters produces a consistently different result, their results do not typically skew the overall average significantly so long as the timing of the various polls is more or less random.
But for some of the national measures we have been plotting recently -- especially Obama's job and favorable ratings and the question about whether Americans perceive things to be "headed in the right direction" or "off on the wrong track" -- a few pollsters that do daily or weekly tracking are producing results with large house effects. Unfortunately that combination, along with the more sporadic timing of other national surveys, is producing the appearance of trends on some charts that are not really trends.
Last night, for example, Andrew Sullivan linked to two charts that appear to show trends in recent weeks: An uptick in the unfavorable rating for Obama and an increase in the percentage saying that things are off on the wrong track. In both cases, unfortunately, the apparent trends are an artifact of timing and house effects.
Let me explain, starting with the right direction/wrong track chart, that follows. (I am using screen shots rather than our live-embedded version here to preserve the look of the chart at the time of this writing -- follow the link to the live chart to use the filter tools yourself):
What Sullivan noticed was the recent uptick in the red line (wrong track) and downturn in the black line (right direction) at the far right (or "nose") of the trend. Now look what happens when we use our filter tool to remove from the trend the two pollsters -- Rasmussen Reports and DailyKos/Research2000 -- whose weekly tracking results provide nearly half (41 of 96) polls plotted in this chart so far during 2009. The recent trend disappears producing an essentially flat line since mid-April:
So removing just two pollsters -- and particularly the two that contributed all four of the poll released in the last two weeks -- eliminates the apparent trend. One problem we have is that these two pollsters release weekly tracks, while the others poll more sporadically. Worse, virtually all of the national pollsters released surveys just before the Obama administration reached its 100th day in office, and we have experienced something of a poll drought since.
But wait. Perhaps those two weekly tracks are catching a more recent trend that we might miss if we rely (for the moment) on the other national tracking surveys that have not produced more surveys in the last few weeks.
To check, let's use the filter tool to select only the surveys from Rasmussen and DailyKos/Research 2000. And just to be safe, I will also turn up the smoothing setting to be especially sensitive to any recent trend:
The trend is almost exactly the same as the version with these pollsters removed, but you can also see that the gap between wrong track and right direction is larger on the second chart of just Rasmussen and Research 2000 (11 points) than on the previous chart excluding those two (4 points), with virtually all of the "house effect" coming from the Rasmussen survey.
So when we look at only the weekly trackers or only the other polls separately, we see flat lines over the last few weeks. When we put them together, we see a recent upward movement on "wrong track." Why? Because when combined the weekly trackers are driving the "nose" of the trend line and the trackers -- especially the Rasmussen track -- is producing consistently different results. So as the Rasmussen results have more influence in the trend line, they tend to drive the red line up and the black line down.
Now let's repeat the exercise with the Obama favorable rating. First, the standard chart showing all surveys. The recent apparent trend is the sharp upward movement on the red "unfavorable" line:
In this case, the Rasmussen and Daily Kos/Research2000 results are six of the seven surveys conducted in the month of May (the new Gallup result was added this morning, after Sullivan's initial post). If we use our filter tool to remove the weekly trackers, the apparent recent change smooths out, reflecting the more gradual increase in Obama's unfavorable rating since the inauguration:
Again, are the trackers picking up a more recent trend that the other national surveys are missing? Here is what the chart looks like if we include only the Rasmussen and DailyKos/Research2000 polls. Here, we see virtually no trend since late March:
The last chart above also clearly shows the enormous house effect separating (in this case) Rasmussen and DailyKos/Research 2000 surveys, with Rasmussen producing consistently lower favorable and higher unfavorable ratings for Obama.
We have discussed the "why" of house effects, especially the consistent differences in the Rasmussen tracking, in previous posts. This case involves something a little more troubling for us: The way house effects and timing have combined to produce misleading "trends" that are more artifact than real. That is something we need to address in a systematic way.
Update: At the suggestion of a reader, Andrew Sullivan removed
only the Rasmussen surveys with similar results to what I obtained above.
By Mark Blumenthal on June 2, 2009 1:18 PM
| Permalink
| Comments (8)
| TrackBacks (0)
By Mark Blumenthal
Too funny to be buried in the "outliers" feature and just in time for New Jersey's Primary Election Day, PolitickerNJ reports (via Hotline Wake-up Call):
In a radio interview tonight, gubernatorial candidate Steve Lonegan called a Fairleigh Dickinson Poll that showed him trailing rival Chris Christie by 24 points in the Republican primary "retarded."
The poll was brought up by NJ 101.5 "Jersey Guys" host Casey Bartholomew, who used it to argue his point that Lonegan was unelectable. When he heard Lonegan use the term "retarded", he checked to make sure he heard correctly.
"I said just that: retarded Fairleigh Dickinson poll," said Lonegan.
See the full article for a gracious response from Fairleigh Dickinson pollster Peter Woolley and the earlier quote from a Lonegan supporter who called characterized the word "retard" as "hate speech."
Our compilation of all public polls on the New Jersey Republican Primary can be found here.
Update: Via @lozzola, a public service link to the "r" word campaign.
By Mark Blumenthal on June 2, 2009 10:36 AM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
How do pollsters determine when an incumbent is vulnerable, especially in a primary? My NationalJournal.com column this week takes up the subject in the context of two recent tests of potential primary contests facing incumbent Senators Arlen Specter and Christopher Dodd.
Two additional thoughts that did not make it into the column:
First, it is not entirely clear (to me) that a classic likely voter screen based on self-reported intent to vote would have produced different results for Joe Lieberman in early 2006 on the initial Democratic primary head-to-head versus Ned Lamont in May of 2006 (I reference both in the column). It may have taken the actual campaign and the awareness it created of Lamont's challenge, to trigger real enthusiasm and intent to vote among the anti-war Democrats that gave Lamont his margin of victory. What might have been useful in early 2006, however, would have been a look the intensity of attitudes about both Lieberman and the Iraq War among Democrats with a true history of primary voting in Connecticut (and not just all registered Democrats). Were hard core primary voters different than other registered Democrats?
Second, it should go without saying, but "vulnerability" is just the first necessary step in defeating an incumbent office-holder. The second and more critical step is a challenger that voters perceive as viable and able that makes a convincing case for why the incumbent should be turned out of office. Many a vulnerable incumbent never faces a truly viable challenger, and many of those that do are able to raise enough doubts about the challenger to win reelection. My guess is that in any given election cycle, there are far more incumbents that we could theoretically describe as "vulnerable" than that ultimately lose.
Update (6/3) - I posted these comment separately, but they bear repeating here:
One reader emailed to take strong exception to my use of the word "misleading" in the following sentence from the column:
One reason for the misleading early numbers in 2006 may have been that Quinnipiac sampled all self-identified registered Democrats rather than a narrower subset of likely primary voters. Their May 2006 sample of 528 Democrats, for example, amounted to 34 percent of the full sample of 1,536 registered voters they interviewed. Yet the actual Democratic primary turnout amounted to just 15 percent of Connecticut's active registered voters.
I will grant that I could have chosen a less loaded word than "misleading," as some will hear it as an insinuation about the pollster's motives or the accuracy of data. For the record, I do not believe that anyone involved in producing the Quinninpiac Poll meant to mislead anyone, and did not mean to imply that the data they reported were inaccurate. The record should show that once they shifted to reporting vote preferences among a narrower group of "likely voters," they showed Lamont running much closer to Lieberman in June, "inching ahead" in July and ultimately leading by a wide margin in early August. Their final poll showed Lamont leading by six percentage points. He won by four -- that's as close as any poll should expect to get.
The larger point I was trying to make with the column is that we mislead ourselves -- and by we I mean all of us, pollsters, journalists, campaigns, political junkies -- whenever we treat samples of a third to half of adults in a state as a meaningful measure of the preferences of "likely primary voters" when the actual turnout is typically a much smaller fraction of adults.
My use of the Quinnipiac Poll was also largely a coincidence. They happened to produce two polls last week with primary head-to-head questions, one in Connecticut, and a similarly designed poll in Connecticut three years ago. However, their practices in terms of sampling primary voters are very similar to those used by most other media pollsters.
By Mark Blumenthal on June 2, 2009 9:23 AM
| Permalink
| Comments (0)
| TrackBacks (0)
May 29, 2009
By Mark Blumenthal
Gallup finds Obama's May approval rating compares favorably to those of previous presidents.
Tom Jensen profiles VA's undecideds and drops a big hint about the partial results from PPP's next VA poll.
Scott Rasmussen sees a Sotomayor bounce.
National Journal's Republican Insiders advise Republicans against blocking Sotomayor (more here).
Joshua Tucker seeks comments from political scientists on the long term effects of supreme court nominees on parties and presidents.
David Hill considers Republican opportunities and challenges with independents; Glen Bolger adds more.
Nate Silver says its a myth that Hispanics oppose gay marriage; Alex Lundry disagrees.
Jay Cost shares first thoughts on Specter v. Sestak.
Chris Bowers looks at Sestak's chances.
Chris Cillizza ponders the gay marriage polling conundrum.
Alex Lundry maps Ed Kilgore's analysis (pdf) of states where 2010 election results could affect congressional redistricting.
Michael Mokrzycki and colleagues post their AAPOR presentation on cell-phone-only voters; Jim Burton has a summary.
Creoleguy32 tweets his experience as a Quinnipiac respondent (may have to scroll down, via @lrainie via @halavais)
And finally, the most. ironic. post. ever.:
"Overall I think the state-by-state electoral college counting is a bit of a distraction."
-- Andrew Gelman, writing on the site known as FiveThirtyEight.com
By Mark Blumenthal on May 29, 2009 4:18 PM
| Permalink
| Comments (0)
| TrackBacks (0)
By Mark Blumenthal
Hopefully, you have already noticed something a little different about our site today. As of this morning, a National Journal banner now sits atop our site, signifying a newly expanded partnership with the National Journal Group, publishers of the National Journal, CongressDaily, The Hotline, The Almanac of American Politics, and NationalJournal.com. While we have partnered since January 2008 -- most visibly through my weekly column on NationalJournal.com -- this new arrangement involves an even closer business relationship.
Some of you may have experienced a few bumps last night as we made the changes, but everything should be working now. If you are experiencing any unusual problems with the site, please let us know.
And as long as we are doing a bit of housekeeping, I also want to take this chance to ask for your input on both what you like about pollster and about the things you dislike or wish we would improve. I have a long list of things I would like to fix, but would appreciate your "qualitative" guidance in setting priorities. So if you can, please take a moment to leave a comment below or email me with your thoughts about the things you would most like to see upgraded or improved.
And thank you all for your continuing support!
By Mark Blumenthal on May 29, 2009 1:45 PM
| Permalink
| Comments (1)
| TrackBacks (0)
