Pollster.com

Articles and Analysis

 

"Statistical Dead Heat?" Depends on Which Statistics You Use

Topics: Barack Obama , Fox News , John McCain

"We have a race that by every measure of every poll is a statistical dead heat. McCain's not supposed to be in this thing, and Obama's supposed to be blowing everybody away and it just isn't happening, at least to this point."

Lou Dobbs (July 17th, Lou Dobbs Tonight)

If you have paid any attention to the news in the past month, you have had a hard time avoiding some journalist or pundit noting that the presidential race is currently a "statistical dead heat" or "essentially tied." The news media, of course, love to cover the horserace aspects of the campaign, particularly in a way that emphasizes how close the election is. But when you step back and gain a little perspective on the big picture, you realize that this race isn't quite the dead heat that it is made to be.

The news media are often a bit myopic in their view of the contest, extrapolating too much from the most recent poll (or even the most recent "surprising" poll). Last week, Fox News released a national survey that showed Obama holding a 41-40% lead, well within the margin of error for the survey. Commentators were quick to emphasize this result and note that the candidates were essentially running neck-and-neck or that the race may even be tied. No doubt there will be a lot of commotion over the latest Gallup/USA Today survey showing McCain ahead (though also within the margin of error) among likely voters. Nevertheless, we gain much better perspective on the state of the race when we look at all available data.

Alan Abramowitz notes that Obama has consistently led in national polls over the past two months. In fact, according to national poll results listed on Pollster.com, Obama had been tied or ahead in 50 consecutive national polls through Sunday. Sure, many polls may show Obama holding a lead within the statistical margin of error, but if Obama and McCain were actually tied, we'd expect as many polls showing McCain ahead as show Obama ahead. Based on some basic calculations, the probability that 50 consecutive national surveys would show Obama tied or ahead if the candidates were actually tied is .0000000000000009. In short, this race is not a "statistical tie," despite what a few scattered surveys (drawing disproportionate attention from the pundits) indicate.

 

Comments
Mainer:

Thank you.

This is a huge source of confusion to some.

One way I have explained this to people is that "margin of error" refers to A SAMPLE, not a collection of samples that were independently drawn.

____________________

boskop:

but it's a squeaker.
it's like the triple crown. if you win by a nose in kentucky: well and good.

if you win by the same nose in the preakness it's looking like you're the winner. that's where we are now.

but the general election lies ahead and the track is longer. it's a different kind of race.

whoever kept tailing you has a better chance to win in the big one when winning by the statistical nose no longer matters. odds on favorites have a statistical habit of fading on the larger course.

and this time around, the GE is a humdinger because the temporal course has stretched out
and around the long track we go.

____________________

Tybo:

extremely relevant when the national vote elects presidents.

____________________

"Based on some basic calculations, the probability that 50 consecutive national surveys would show Obama tied or ahead if the candidates were actually tied is .0000000000000009"

This is uncharacteristically sloppy statistics. Were you able to operate in this way, a pollster could take his/her n=1500 poll, break it into two n=750 polls, and say "Obama is ahead in both by x standard deviations. The chance of them both being x standard deviations off is equal to (chance poll A is off) times (chance poll B is off)." But standard deviation of the mean simply does not work this way.

____________________

An addition to the above: I feel I should say both that I am an Obama supporter and that I agree with the premise of the article. I just think that ".0000000000000009" implies a level of precision not actually backed up by the data.

____________________

schen:

I think you're looking at this wrong amarquis. I would imagine the author simply used the binomial probability of 50 consecutive occurrences of an event with a 50-50 chance of the event happening or not. Standard deviation is not necessary at all in this calculation, hence "basic calculations."

____________________

Ah, schen, I didn't see that it was simply 0.5^50. Regardless, the number still implies a precision not contained in the data, for many reasons. For example, since a large number of the consecutive polls are rotating window polls, you are effectively counting many very intertwined polls as independent events.

Like I said, I agree with the sentiment, just not the statement.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR