Mark Blumenthal | June 18, 2009
Walter Mebane, the University of Michigan political science and statistics professor who specializes in statistical tools "for detecting anomalies and diagnosing fraud in election results," has updated his assessment of the official vote return statistics for the Iran elections. Mebane now says he sees "moderately strong support for a diagnosis that the 2009 election was afflicted by significant fraud."
In his initial analysis, Mebane used town-level data from the second "run-off" stage of the 2005 Iranian elections to model expectations for the 2009 results. The technical difference in the update is that Mebane has incorporated town-level data from the first stage of the 2005 elections. In his revised analysis, Mebane is struck by "the large number of outliers":
One might expect that given the increased political resolution provided by having measures of the ﬁrst-stage candidates' support, combined with the turnout ratio variable interactions, the model would do a good job capturing more of the variations in the 2009 vote
His conclusion. Something is fishy in the official 2009 results and the deviations appear to benefit Ahmadinejad:
More than half of the 320 towns included in this part of the analysis exhibit vote totals for Ahmadinejad that are not well described by the natural political processes the model of Table 15 represents. These departures from the model much more often represent additions than declines in the votes reported for Ahmadinejad. Correspondingly the poorly modeled observations much more often represent declines than additions in the votes reported for Mousavi.
Modiﬁed conclusion: In general, combining the ﬁrst-stage 2005 and 2009 data conveys the impression that while natural political processes signiﬁcantly contributed to the election outcome, outcomes in many towns were produced by very different processes. The natural processes in 2009 Ahmadinejad have him tending to do best in towns where his support in 2005 was highest and tending to do worst in towns where turnout surged the most. But in more than half of the towns where comparisons to the ﬁrst-stage 2005 results are feasible, Ahmadinejad's vote counts are not at all or only poorly described by the naturalistic model. Much more often than not, these poorly modeled observations have vote counts for Ahmadinejad that are greater than the naturalistic model would imply. While it is not possible given only the current data to say for sure whether this reﬂects natural complexity in the political processes or artiﬁcial manipulations, the numerous outliers comport more with the idea that there was widespread fraud than with the idea that all the departures from the model are benign. Additional information of various kinds can help sort out the question. Remaining is the need to see data at lower levels of aggregation and in general more transparency about how the election was conducted.
Having watched a lot of misleading exit poll pseudoscience ricochet around the internet in the aftermath of our own elections in 2004, I have a reflexive caution about quick blog posts claiming statistical evidence of fraud. Walter Mebane falls in an altogether different category. No one is better qualified to find statistical evidence of fraud in election data. He has made his raw data and R-code available (here), and other statisticians (including those with better knowledge of Iran's elections) may reach different conclusions. But if someone like Walter Mebane is no longer on the fence about the 2009 data, it means a lot.
Update - Washington Post polling director Jon Cohen reminds us of the limits of this sort of circumstantial evidence:
This analysis adds a new dimension to the debate over the results, but is still well short of "hard evidence" of fraud, particularly given our limited understanding of voting behavior in Iran.
He also summarizes a Post letter-to-the-editor that makes a point I tried to make (albeit less clearly) on Tuesday:
And all of this may miss a key point brought up by a reader in today's Post: in a letter-to-the-editor, John Cronin of Takoma Park writes that our search for "proof" through numbers may be misguided. "[W]hen an unelected ayatollah -- the "supreme leader," no less -- controls much of the media, the military and the courts, the whole state is effectively rigged," Cronin writes, "[i]t's hard to imagine any election being truly fair under such conditions, regardless of the extent to which the ballot boxes are stuffed."