Guest Pollster | October 22, 2008
Alex Lundry is the research director at the Republican polling firm, TargetPoint Consulting.
With only two weeks remaining, pollsters and journalists alike have rightly been reexamining the legitimacy of their polling numbers. In particular, there have been caution flags regarding three phenomena that may be unaccounted for in many of the public polling numbers: 1) the difficult to poll cell-phone only voter, 2) a possible surge in youth and minority turnout missed by likely voter models, and 3) the Bradley-Wilder Effect causing artificially deflated numbers for McCain.
Unfortunately, poll-watchers today have another reason to be wary: a likely over-report of voter registration, especially among African American voters, possibly causing surveys of both registered voters and likely voters to overstate support for Barack Obama.
A 2007 article in Public Opinion Quarterly (link, gated) by Andrew Fullerton, Jeffrey Dixon, and Casey Borch, looked specifically at the problem of registration over-reporting - in which unregistered respondents inaccurately state they are registered voters. Their analysis relied upon National Election Study (NES) validation studies between 1976 and 1980 (the most recent year for which both registration and voter validation data are available - an analytical shortcoming the authors freely admit to). Seeking the drivers of this behavior, they found that blacks are more likely to overreport their registration (along with those that are better educated, live in the "Deep South", and have strong partisan beliefs). The implications of this are particularly relevant to this year's election polls, as the authors detail in this critical point:
If the level of registration overreporting is comparable today, as we believe it is, this subpopulation inflates the number of potential voters in pre-election surveys because they are typically based on samples of self-reported registrants. More importantly, if our finding that blacks are more likely to overreport registration than voting holds true today, as we think it may, this could skew the results of pre-election surveys, likely in favor of a Democratic candidate given blacks' historical affiliation with this party.
If ever there was an election in which black respondents felt a social desirability bias to over-report their registration, this would be it. Support for Barack Obama among African Americans is nearly monolithic, and we are treated to frequent numeric and anecdotal accounts of increased enthusiasm and engagement among the black community. A reasonable person would conclude that an unregistered African American, called to participate in a survey, would feel some sort of pressure (either known or unknown) to say that he or she is indeed registered, and continue with the survey.
How significant could the bias be? Word of increased registration and enthusiasm among African Americans makes it difficult to assign a precise number, but the data itself can at least provide us with a guidepost: between 1976 and 1980, 11% of NES respondents overreported their registration. Among this group, two-thirds (7% of all respondents) later claimed in a post-election study to have actually voted.
These findings should lead us to be especially wary of recent polling in Georgia and North Carolina showing Barack Obama within striking distance of John McCain, as Fullerton et al.'s analysis indicates that residence in the "Deep South" - states with the heaviest concentration of blacks - also makes a meaningful difference in registration over-reporting (though, to be fair, North Carolina is considered a "Peripheral South" state in their treatment). Still, one way to mitigate this problem - voter registration based sampling - is used by Insider Advantage, a frequent pollster in Georgia, as well as PPP, which has been active in North Carolina.
Still, as much as these findings intuitively "make sense" there are a number of reasons to be skeptical: first, the authors themselves point to a number of issues with their analysis (old data, problems with the validation of African Americans' registration records, etc.), and second, the very reasonable assumption that even if this effect did exist, it could be cancelled out if African Americans turnout at higher rates than pollsters predict they will.
Despite these limitations, Fullerton et al.'s analysis should give pollsters and poll consumers sufficient pause as they read the inevitable flood of horserace results over these remaining weeks.