Pollster.com

Articles and Analysis

 

Novak and "The Bradley Effect"

Topics: 2008 , Barack Obama , Bradley , Bradley/Wilder , Exit Polls , Hillary Clinton , John Zogby , Kathy Frankovic , Robert Novak

Robert Novak's column last week led with this reference to the Pennsylvania exit poll results:

When Pennsylvania exit polls came out late Tuesday afternoon showing a lead of 3.6 points for Hillary Clinton over Barack Obama, Democratic leaders who desperately wanted her to end her candidacy were not cheered. They were sure that this puny lead overstated Obama's strength, as exit polls nearly always have in diverse states with large urban populations. How is it possible, then, that Clinton, given up for dead by her party's establishment, won Pennsylvania in a 10-point landslide? The answer is the dreaded "Bradley effect."

Prominent Democrats only whisper when they compare Obama's experience, the first African American with a serious chance to be president, with what happened to Los Angeles Mayor Tom Bradley a quarter-century ago. In 1982, exit polls showed Bradley, who was black, ahead in the race for governor of California, but he ultimately lost to Republican George Deukmejian. Pollster John Zogby (who predicted Clinton's double-digit win Tuesday) said what practicing Democrats would not: "I think voters face to face are not willing to say they would oppose an African American candidate."

Unfortunately, Novak confounds two issues, and Zogby's contribution confuses things further. The "Bradley effect" (also called the "Bradley/Wilder effect," the latter based on the 1989 election of Doug Wilder in Virginia by narrower margins than indicated by pre-election polls) pertained less to exit polls but to pre-election telephone surveys. The underlying theory was that white respondents were sometimes unwilling to reveal their preference for the white candidate in a bi-racial contest when they felt some "social discomfort" in doing so. That is, respondents would be less likely to reveal their true preference in a telephone interview if they believed the interviewer supported a different candidate. The most important evidence was an observed race-of-interviewer effect: Support for Doug Wilder in one 1989 survey (pdf) was eight points higher when the interviewer was black than when the interviewer was white.

The problem with extending this idea to the 2008 exit polls is that -- contrary to the apparent assumptions of both Bob Novak and John Zogby -- exit polls do not involve a "face to face" interview. Rather, the exit poll interviewer's task is to randomly select and recruit respondents, hand them a paper questionnaire, a pencil and a clipboard and allow the respondents to privately fill out the questionnaire and deposit it into a large "ballot box."

ballot box.jpg (JPEG Image, 750x123 pixels).png

The more likely explanation for the consistent Obama skew in the exit polls this year is likely less about "voters not willing to say they would oppose an African American candidate," than about the relative youth of the interviewers, and the well established problem that the typically younger exit poll interviewers have in winning cooperation from older respondents. Here is a summary I wrote two years ago about information included in the official, post-election report on the 2004 exit polls:

The [National Election Pool] NEP exit polls depended heavily on younger and college age interviewers. More than a third (36%) were age 18-24 and more than half (52%) were under 35 years of age (p. 43-44). These younger interviewers had a much harder time completing interviews: The completion rate among 18-24 year olds was 50% compared to 61% among those 60 or older. The college age interviewers also reported having a harder time interviewing voters...The percentage of interviewers who said "the voters at your location" were "very cooperative" was 69% among interviewers over 55 but only 27% among those age 18 to 24 -- see p. 44 of the Edison/Mitofsky report.

Given the huge differences by age in both pre-election and exit polls -- Obama wins those under 30 while Clinton dominates among those over 60 -- an age-related selection bias is not surprising. And the issue may not be about simply getting the age mix right in the exit poll. The issue may also be related to the "social discomfort" theories behind the Bradley-Wilder effect.

Respondents may be making judgements about the exit poll interviewers based on their appearance (age, gender and race) that influence whether they agree to participate or avoid the interviewer altogether. Similarly, while exit poll interviewers are supposed to be carefully counting exiting voters and sticking rigidly to instructions that they select every fourth voter (or whatever interval they are instructed to select) anecdotal evidence suggests that those with less experience often deviate from the procedure and "take who they can get." So less experienced, overburdened interviewers are probably making judgments about which respondents (based on their age, gender and race) might be most likely to cooperate.

Again, quoting from my own summary two years ago:

"It's not that younger interviewers aren't good," as Kathy Frankovic puts it (slide #30), "it's that different kinds of voters perceive them differently." Put all the evidence together, we have considerable support for the idea "that Bush voters were predisposed to steer around college-age interviewers" (Lindeman, p. 14) or, put another way, that "when the interviewer has a hard time, they may be tempted to gravitate to people like them" (Frankovic, slide #30).

It is not at all surprising that this same mix of issues -- younger interviewers who have trouble winning cooperation with older respondents and a huge age differential in the results -- produces a consistent skew to Obama in the context of 2008.

 

Comments
jsh1120:

Thanks, Mark. Novak's agenda in his columns is obvious. And the "Bradley/Wilder effect" has received growing attention among the mainstream media, many of whom have no real idea what it is. Add to that the tendency of exit polls to validate results rather than to examine the composition of the support coalitions of the individual candidates and there's a near perfect storm of misunderstanding about the issue.

____________________

gomer0:

Obama has been either getting in his election results what he was polling pre-election (+/- 1 or 2% of the realclear average - NH, Ohio, Texas, PA) or he exceeds them (WI, SC, VA, etc).
Ironically, if you look at the last Field Poll in the Bradley race or the last Mason-Dixon poll in the Wilder race, both candidates received basically the same percentages that they received in pre-election polling. There was no Bradley effect in the Bradley or Wilder race! All the "undecideds" went for their opponents just like most went to Clinton.

____________________

americaferreraisafatho:

one word :

Wisconsin

____________________

Ciccina:

You've been fighting the good fight, Mark, but I'm afraid this train has left the station.

The notion that a significant number of white Democrats aren't voting for Obama because they are racist is just too convenient to go away. It allows pundits to explain away Clinton's support and negate discussion of her positive qualities. At it gives pundits who have been wrong about Clinton's level of support the opportunity to save face. Its not the pundits misreading the voters, you see; its those nasty voters hiding their true proclivities.

Plus, it plays into the 'Clintons-as-racist' narrative. In short, this meme is too useful to die.

I find this topic grating not just because of my own strong candidate preference, but because I spent many, many good hours working with pollsters to try to figure out how to measure the impact of my group's issues on voters.

This is the holy grail of third party electoral groups: proof that it was your particular position on your particular issue that made the difference for the winning candidate.

Knowing how difficult it can be to get at one specific motivating factor, it really toasts my challah when someone looks at one or two figures from an exit poll and divines that a convoluted interplay of racism and shame led to a certain result.

It would be great to see polling that really tries to get at the role of race and gender in candidate preference right now. I know there have been previous posts on this but it would be great to see further serious consideration of the sort that would only appear here.

____________________

Patrick:

What some of you are overlooking is that each state is different and somewhat unique demographically. Even states that seem similar in demographics are often very different. For example, both NM and FL have a high % of Hispanics. But Cuban Americans don't identify culturally with Mexican Americans, so you cannot say that the Hispanic populations of those states will back the same candidate. Some "traditional" states like Iowa or Utah have generally voted more sexist than racist; others the reverse. What applies to WI or SC or VA doesn't apply to OH or PA or FL. This Bradley effect is likely much more of an issue in states like PA, OH, MI, and MA where the REALITY is that there are a higher percentage of populations that will not vote for an African American than more "progressive" states. I lived in MA for many years. While it is certainly a "progressive" state overall (i.e. the only state McGovern carried), there is tremendous racism there. Unfortunately for Obama, the REALITY is that many of the states that have the highest populations of the demographic groups he can't seem to connect with and/or racists are the very swing states that decide presidential elections electorally. You see it already in the match-up polls in such states as MA, FL and OH. The REALITY is Obama could win 98% of the African American vote and 80% of yuppies and under-30 voters nation-wide and he'd still lose OH, PA, FL and therefore, the entire election. In this regard, Obama currently isn't even as electable as Kerry or Dukakis. If Swiftboat ads and Willy Horton ads were able to sink them (when they started with a big lead in the polls), just imagine what the Republicans and 527's will do to Obama with Rev. Wright ads in every state (esp. when they're starting the race virtually tied). Just watch.

____________________

Even an exit poll conducted on Election Day showed Wilder winning by 10 points, while accurately tallying the vote in the other two statewide races. Unlike most exit polls that use an anonymous written ballot to collect voters' responses, this one had interviewers asking voters face-to-face how they voted, a situation that might increase the pressure to provide a socially desirable response.

Sometimes exit polls have been face to face, though nowadays not so.

ttp://pewresearch.org/pubs/408/can-you-trust-what-polls-say-about-obamas-electoral-prospects

____________________

Nick Panagakis:

Excellent piece Mark.

1. I think it's amusing that all of these people leaked premature exit poll survey numbers that were contradictory due to varying estimate methods. FINAL exit poll vote estimates are weighted by actual vote returns which, of course, are not available at 5:30 PM. Prior estimates are likely to be contradictory for single digit margin races.

2. Robert Novak �does not understand the process�. Exit poll respondents do have an opportunity to answer the questionnaire privately. I did exit polls for 25 years labeling questionnaires �Secret Ballot� which works as well as �Confidential�. This is instructive for rare instances when a couple go to the polling place together and the wife does not want to reveal her vote. BTW, I don�t know any women like that.

3. Bradley Effect. Correct me if I am wrong hasn�t but this been discredited over the years in other elections? About 20 years ago this was the theory in a VA exit poll when Doug Wilder was the candidate, 55% versus 50.1% outcome. However, it was impossible to separate interview effects from sample precinct selection/weighting error.. This year, return of the Bradley effect was cited in a NY Times op-ed piece after regarding NH phone polls. We have polled many black candidate races in IL, Chicago, and Cook County with no such consistent effect. This includes primary and general election exit polls that Harold Washington won by 2-3 points in 1983.

4. This is really interesting: "end-of-day exit polls have shown a consistent skew favoring Obama". I have heard this before. Mark will remember the 2004 Mitofsky-Lenski analysis of their state exit poll results that concluded �differential participation�; i.e., Democratic candidate voters were more likely participate in exit poll interviews than those voting for Republicans.

Is there differential participation here Mark? Are younger, more educated Obama voters more likely to participate than older, less educated Clinton voters? Voter demographics may be a factor here: older, less educated voters are also less likely to participate in exit polls. And are exit polls sponsored by national media shunned by Clinton voters because of what they believe is more favorable coverage of Obama much like the Republican voters in 2004?

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR