Pollster.com

Articles and Analysis

 

Lundry: A Different Kind of Bradley-Wilder Effect?


Alex Lundry is the research director at the Republican polling firm, TargetPoint Consulting.

With only two weeks remaining, pollsters and journalists alike have rightly been reexamining the legitimacy of their polling numbers. In particular, there have been caution flags regarding three phenomena that may be unaccounted for in many of the public polling numbers: 1) the difficult to poll cell-phone only voter, 2) a possible surge in youth and minority turnout missed by likely voter models, and 3) the Bradley-Wilder Effect causing artificially deflated numbers for McCain.

Unfortunately, poll-watchers today have another reason to be wary: a likely over-report of voter registration, especially among African American voters, possibly causing surveys of both registered voters and likely voters to overstate support for Barack Obama.

A 2007 article in Public Opinion Quarterly (link, gated) by Andrew Fullerton, Jeffrey Dixon, and Casey Borch, looked specifically at the problem of registration over-reporting - in which unregistered respondents inaccurately state they are registered voters. Their analysis relied upon National Election Study (NES) validation studies between 1976 and 1980 (the most recent year for which both registration and voter validation data are available - an analytical shortcoming the authors freely admit to). Seeking the drivers of this behavior, they found that blacks are more likely to overreport their registration (along with those that are better educated, live in the "Deep South", and have strong partisan beliefs). The implications of this are particularly relevant to this year's election polls, as the authors detail in this critical point:

If the level of registration overreporting is comparable today, as we believe it is, this subpopulation inflates the number of potential voters in pre-election surveys because they are typically based on samples of self-reported registrants. More importantly, if our finding that blacks are more likely to overreport registration than voting holds true today, as we think it may, this could skew the results of pre-election surveys, likely in favor of a Democratic candidate given blacks' historical affiliation with this party.

If ever there was an election in which black respondents felt a social desirability bias to over-report their registration, this would be it. Support for Barack Obama among African Americans is nearly monolithic, and we are treated to frequent numeric and anecdotal accounts of increased enthusiasm and engagement among the black community. A reasonable person would conclude that an unregistered African American, called to participate in a survey, would feel some sort of pressure (either known or unknown) to say that he or she is indeed registered, and continue with the survey.

How significant could the bias be? Word of increased registration and enthusiasm among African Americans makes it difficult to assign a precise number, but the data itself can at least provide us with a guidepost: between 1976 and 1980, 11% of NES respondents overreported their registration. Among this group, two-thirds (7% of all respondents) later claimed in a post-election study to have actually voted.

These findings should lead us to be especially wary of recent polling in Georgia and North Carolina showing Barack Obama within striking distance of John McCain, as Fullerton et al.'s analysis indicates that residence in the "Deep South" - states with the heaviest concentration of blacks - also makes a meaningful difference in registration over-reporting (though, to be fair, North Carolina is considered a "Peripheral South" state in their treatment). Still, one way to mitigate this problem - voter registration based sampling - is used by Insider Advantage, a frequent pollster in Georgia, as well as PPP, which has been active in North Carolina.

Still, as much as these findings intuitively "make sense" there are a number of reasons to be skeptical: first, the authors themselves point to a number of issues with their analysis (old data, problems with the validation of African Americans' registration records, etc.), and second, the very reasonable assumption that even if this effect did exist, it could be cancelled out if African Americans turnout at higher rates than pollsters predict they will.

Despite these limitations, Fullerton et al.'s analysis should give pollsters and poll consumers sufficient pause as they read the inevitable flood of horserace results over these remaining weeks.

 

Comments
KS Rose:

This makes some sense but I'm left wondering if these voters believed they were registered but were not. I'd really like to end 'voter registration'. It should be a national holiday, with all same day sign up, you bring whatever paperwork you'd use to show where you live and be allowed to vote once and only once. This might have been impossible inn the 1950's but it's certainly not impossible now.

____________________

nattyish:

This is a valid and interesting point of caution regarding how likely voter screens operate: clearly, voters who self report that they voted four years ago and are registered for this year when they are not can alter likely voter models.

However, as the author points out, registered voter sampling should eliminate that effect. As a result, we should see different results from pollsters (PPP and Insider Advantage) who sample from RVs rather than random number dialing.

A quick look at recent Georgia polls shows Insider Advantage actually running two to three points better for Obama rather than McCain, the opposite of what this article would indicate. Meanwhile, PPP has been consistently giving Obama his best results in NC of any of the pollsters there, by three to four points.

While two pollsters don't provide a definitive refutation, and the vagaries of individual LV models can adjust things massively (PPP trends maybe half a point D, Insider Advantage might be that much towards the R), we definitely don't see any clear evidence for this effect by comparing pollsters.

____________________

brian d:

funny - paralleling a respondent's answer to voter registration to church going... Its an interesting notion but I'm skeptical about how big an impact this could really actually have.

____________________

mattn:

This is quite interesting, but how does it jibe with the results from the Democratic primaries, especially in states like South Carolina, where IIRC the polls *underestimated* the black vote for Obama? It seems to me that this hypothesis is somewhat contradicted by the empirical results.

____________________

brambster:

Wouldn't the huge AA early voting turnout in states like NC and GA suggest that this isn't the case??? I'm not sure where those numbers come from, but it makes sense that there would be the most enthusiasm among the AA population this cycle and this should turn into real votes just like enthusiasm carried Obama to a win in all of the traditional-style caucuses.

I don't doubt for a second the likely disproportional over-reporting of registration being heavier in certain groups that are the most enthusiastic. I also however believe that AA's are less likely to take a pollster's call than some other groups, and the poll results have often found an obvious over-representation of McCain allegiances of AA voters polled. I expect Obama to get 93% of this vote as opposed to 88% that went for Kerry in 2004. Harold Ford received 95% in his Senate bid on turnout that was comparable to the presidential election year. Essentially there seems to be a counter effect for every effect claimed. There are a lot of things that point to a huge turnout this cycle with that extra turnout coming from groups overwhelmingly supporting Obama.

I strongly believe that young voters and AA's will increase dramatically this cycle due to both enthusiasm and targeted registration drives. I believe that states like NC and VA will likely surprise well to the upside for Obama come election day just like he generally outperformed the polls in primaries with AA populations larger than 15%, and these states also have large young professional populations mostly re-located from cities in the Northeast.

I don't see poll over-performance happening in places like Ohio and Florida however since those populations have been pushed heavily for two cycles now and there isn't much about the electorate that will be changed in those states.

If the Bradley effect were true, McCain wouldn't be doubling down on PA where he is generally about 10 points behind, but is the only state that has enough EV's to overcome losses in CO and VA. McCain's campaign knows they are beat in those states, and they surely have tested carefully for such effects as generally bad polling. This isn't shaping up to be a 3 point election, it's shaping up to be a 10 point election.

____________________

Vicente Duque:


Other effect to add to your preoccupations with cell phones, party weighting, bradley effect, lazy turnouts, etc ... :

Look how negative are Mainstream American Newspapers towards McCain. And Foreign Newspapers are scared to hell of a McCain Triumph.

I think that the power of Television is exaggerated, we watch a lot of it, but we don't believe a single word of the eternal repetions and low information.

And Newspapers, Press Journalism is underestimated and undervalued. Those that read influence strongly others on election day and previous days.

A victory of McCain is possible but that would create a problem of governability. Not a revolution but an Administration hated even more than the Bush Administration, by many people inside and outside America.

That would be a Big Hurdle in International Relations.

Bush is sweet and friendly compared to McCain, Bush is mistaken but well intentioned. Bush is tractable. McCain may be not.

http://tossUpStates.blogspot.com/

http://milenials.blogspot.com/

Vicente Duque

____________________

countrymous3:

I like nattyish's suggestions for testing this!

Importantly, increased over-reporting of registration by a sub-group doesn't necessarily lead to a bias in the results, as long as all groups over-report to some extent. Consider the following *toy example* of a state that is split 80/20 White/Black.

20% Blacks overreporting registration by 5%, = 1% of sample, almost all going to Dems.
-> ~1% Dem Shift

80% Whites overreporting registration by 2% = 1.6% of sample going 60-40 to Reps
-> ~1% Rep Shift


To assess the possible importance of this phenomenon, we need to see some effect sizes for the different subgroups...

____________________

StandardDeviation:

This is not terribly informative. Bias of this kind has very limited impact; in order for it to have an impact, the unregistered AA "voters" would need to support Obama more so than truly registered AA voters. Given that AA voters in general already seem to support Obama 90-10 or more (and there is ample data to estimate the proportion of AA supporting Obama now), it seems unlikely that there is a large discrepancy in the voting preferences of unregistered AA and registered AA. Indeed, one could conduct a poll without sampling AA at all and simply take a conservative approach that 8 of 10 will support Obama.

A much more useful endeavor is predicting/weighting the % of AA voters relative to all voters. Different polls have different weightings for AA turnout, Dem/Repub split, youth vote, etc...and this, more than anything will impact the accuracy of the poll.

____________________

Observer:

On the question of extra registered voters I think it is often overlooked that registration drives are not new. They were a big feature four years ago and doubtless before that. It is therefore necessary to run pretty fast just to stay still. The benefit will only come by the margin that current efforts have improved on what went before.

And a problem with any assessment of that is we don't know whether any significant percentage of new registrations are non-people, re-registrations used to pad out quotas by canvassers.

On the question of more young voters we all know that people are living longer and the average age of the population is going up. Once again therefore an improved turnout of young voters is needed just to maintain their percentage contribution to the overall vote, i.e. this time a question of having to run faster just to stay still.

____________________

JedRothwell:

brambster asks:

"Wouldn't the huge AA early voting turnout in states like NC and GA suggest that this isn't the case??? I'm not sure where those numbers come from . . ."

The numbers come from official sources and they are completely reliable. Georgia is required to keep track of how many people of different races vote, under the Voting Rights Act.

____________________

wjbill49:

wow Vicente gives Bush a lot of undeserved credit for being an idiot

____________________


Friend wjbill49 :

I see some stress on the face of Mr Bush, and his voice shows some emotion of sadness.

I sincerely believe that Mr Bush has many doubts about his foreign policies and has tried to correct course the best he can.

George W. Bush may have some similarities to Lyndon Baines Johnson who also had trouble with another country Vietnam. Johnson was very much depressed in later life, but I do not know the causes.

Jimmy Carter said that losing an election is extremely painful. I see signs of stress and depression in Sarah Palin. She mentions the word "depression" very often and it is not the economy. Her cheer and joyfulness may be beauty queen training.

The madness of McCain and his rage may be the first symptons of realizing that it is very difficult to win.

Psychiatrists are reporting that McCain is a survivor and as a jet fighter pilot he has to be a daring, impetuous, impulsive, rash guy. Rage and wrath are wonderful for those pilots in dog fights.

He now has to eject his seat and open his parachutes.

Vicente

____________________

bmcfar01:

"wow Vicente gives Bush a lot of undeserved credit for being an idiot"

This flippant comment aside, he makes a good point when he states that at this point, a McCain upset, come back, miracle win, what have you (depending on your affiliation) would be EXTREMELY distressing to Democrats and certain members of the media.

A McCain victory maybe so "unacceptable" to Congressional Democrats that they probably would stonewall and deliberately stall any McCain administration initiatives.

Rightly or wrongly, we're getting to the point of no return in which a McCain victory would plunge the country into a different sort of crisis.

____________________

douglasdao:

This article sounds like the coming cover story to explain Republican voter suppression. The GOP is doing everything it can to deny minorities the right to vote. When they succeed, they will call it "over-report of voter registration, especially among African American voters, possibly causing surveys of both registered voters and likely voters to overstate support for Barack Obama." What it really is is the Republicans trying to steal another election.

____________________

bmcfar01:

"GOP voter suppression"

Just like they suppressed Obama's votes in NH, CA, MA, OH, TX and threw the election to Clinton during the primaries. Right?

Hey, I'm a Democrat and will probably vote for Obama, but this caricature of Republicans as bent on suppressing millions and millions of votes is just too far fetched. I'm sure voter suppression occurs, I'm sure it can effect a few precints, but you're talking about millions of votes being lost, thrown away or switched on election day.

That's just not going to happen.

Obama over performs in polling. It remains to be seen if that trend continues to election day.

____________________

kglore in PA:

Recent CNN reports of early voting in West Virginia have already noted electronic touch screen voting machines that are flipping votes from democrat to republican when the box is touched. Eye witnesses have repeated their vote only to find the machine continuing to flip their vote. The machines are made by ESS and they happen to be the same machines implicated in vote flipping in Ohio in 2004. This must be investigated by the proper authorities and this must be stopped before election day. There are currently 97,000 of this company's machines being used in this election, so GOP vote fraud is alive and well and when republican pollsters offer cover for why the vote may be closer than the polls suggest, will anyone question a come-from-behind McCain victory. We can't let this happen!

____________________

countrymous3:

@bmcfar01
I don't think anyone is claiming that Republicans engaged in voter suppression during the Dem primaries.

But there is plenty of evidence that R's do attempt to discourage voting among predominantly D groups (urban precincts, new voters). It should come as no surprise that they're going to do everything they legally can (lodge protests at polls in Dem areas, attempt to purge voter rolls on technicalities) in similar areas.

Here's a CNN article about a gentleman who was just released from jail for jamming Democratic phone lines on election day in 2004:
http://www.cnn.com/2008/POLITICS/10/22/rigging.election

Was he a 'lone wolf'? The evidence suggests otherwise:
http://www.salon.com/news/feature/2008/10/22/voter_supression_guide/

____________________

voteforamerica:
____________________

SnowMore:

Pollsters weight their samples by race. they know the racial breakdown of the registered and likely voters, and they make sure their polling data reflects these numbers.

Thus, the only way this effect will make a difference is if the opinions of unregistered African Americans differ significantly from those of registered AA voters. That seems highly unlikely.

____________________

Chicago:

Same thing with newly registered voters and young voters... in early voting they are not showing up. This is why Obama will not win, his support in polls is not the same as votes, and all the websites and pollsters are forgetting this. Its not who says they support you it is who votes and who counts the votes. Republican shananigins will take care of that after they turn voters away with the long lines and ID challenges.

____________________

What an exciting topic!
I need to Buy Viagra right away

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR