Harry Enten | July 22, 2010
Topics: Accuracy , Georgia , Primary elections
Polling critics are fast to point out when polls get it wrong, so let us point out an instance of the polls getting it right. In Tuesday night's Georgia Republican gubernatorial primary, Karen Handel and Nathan Deal placed first and second to advance to a runoff on August 10th. The Pollster.com aggregate did a good job of catching the rise of both these candidates and fall of one time frontrunner John Oxendine, and the individual polls were pretty good too.
In the final seven days before the primary, five polls came out and all five had Handel finishing in first place. Four of the five correctly predicted that Handel and Deal would end up in the runoff. Three of the five forecasted Handel first and Deal second. The final two polls taken accurately projected first through seventh. Most impressively, the final poll (conducted by Insider Advantage) came within three percent of pegging each difference in percentage of the vote between the top four candidates. Why is all of this impressive to me?
First, the only poll (Mason-Dixon) that incorrectly put John Oxendine in the runoff was also the only live interview poll in the field during the final week. Many news organizations still refuse to share results from automated phone (or dismissively referred to as robopolls) because of such fears as five year old (or even a dog) could theoretically be interviewed. All of the "less reliable" automated polls got the final the runoff participants correct, but the supposedly "scientific" poll did not.
Second, southern and gubernatorial primaries have been shown to be the toughest statewide contests to poll. The fact that the race was in fluctuation in the final weeks and the polls were able to pick up on a clear trend toward Handel and Deal and away from Oxendine is especially striking with this past performance in mind.
Third, the pollsters that were furthest off the mark were the ones who have the best historic track record. Mason-Dixon & Rasmussen have been two of the top ten pollsters over the past twelve years, but both of them missed the correct order of the top two finishers. It is important to point however that Oxendine's (and Rasmussen's Deal) percentage was just outside of the margin of error when undecideds are allocated based on those who have registered an opinion.
Fourth, the most accurate poll was conducted by Insider Advantage, which surprised me because of their lack prior lack of transparency and poor performance. Earlier this year, I was involved with a tussle with Insider Advantage. Why? I simply wanted to find out whether their Insider Advantage / Florida Sun-Times poll was live interviewer or automated phone for an article I was working on related to the Florida Republican Senatorial primary. Insider Advantage did not clearly state either way on their website. I repeatedly tried to contact Insider Advantage's CEO Matt Towery for the article, but did not get a response. Florida Sun-Times' political reporter David Hunt, who had written articles based on the poll, believed the poll was "live" interviewer. Only after Mark Blumenthal send an inquiring did I found out that Insider Advantage employs an automated phone technique. In addition, Nate Silver found them to be the second worst pollster with at least ten polls conducted over the past twelve years. Last night, none of these past actions seemed to matter as Insider Advantage nailed the placement of the seven candidates and difference of vote between the top four Republican candidates for governor.
Overall, It was a good, but odd night for polling where up was down and down was up.