Pollster.com

Articles and Analysis

 

Update: Cook County Exit Poll

Topics: Chicago Current , Cook County , Exit Polls , National Election Pool (NEP) , Voter News Service (VNS)

Another update, this one on the volunteer exit poll conducted this week in Cook County Illinois by the recently launched Chicago Current. Current editor Geoff Dougherty posted a refreshingly candid postmortem on their efforts:

At 6:11 p.m. yesterday, before the polls closed, I wrote that our exit polling suggested Toni Preckwinkle had the Cook County Board president's race locked down.

And I was right. Our survey honed in on Preckwinkle's strong performance early in the day, and continued to highlight her lead as the election progressed.

And yet ... our poll was wrong. I predicted Preckwinkle would snag 69% of the vote, and noted that the poll had an 8% margin of error. Preckwinkle ended the day with 49% of the vote -- well outside that margin.

Such are the joys and pains of exit polling.

There's more, and it's worth clicking through to read the rest.

I would give the Current an "A" for effort and transparency, but we need to be realistic about the quality of the survey they ultimately produced. Dougherty says it cost just $200, "most of which went for a $100 rental car," and don't think he would argue with the conclusion that they got what they paid for. The poll managed to collect just 93 completed interviews at only 9 of 25 precincts (presumably) selected at random. As Dougherty reported at 1:32 p.m. on Tuesday:

So far we've got about 30 responses. We'll be taking a pause here as our field crew relocates to new spots and starts talking to voters.

We'd originally planned to survey 25 precincts, but logistics are interfering, and we'll probably wind up with about half that. We'd targeted 600 voters, but low turnout will probably leave us with about half of that count.

Never mind the very small sample size. How truly random was the sample? It's hard to tell from this description, but the execution clearly fell short of ideal.

Dougherty says that the "networks often pay tens of thousands of dollars for these things." That's not quite right. I'm not sure how it translates into a per-state cost, but the every-two-year National Election Pool (NEP) exit polling operation has a multi-million dollar budget (Voter News Services, VNS, the forerunner to NEP, operated in 2000 on a budget of over $35 million; my understanding is that current costs are much lower but still in the millions). Note that in most states of interest, NEP will sample 20 to 50 precincts. As the scale of what the Current was attempting in a single county was in line with the exit poll that NEP conducts in each state.

I write this post not to beat up on the Current -- again, I give them credit for enterprise and transparency -- but to remind my media colleagues that all "exit polls" are not created equal. Not by a long shot.

Update: The cost statistic I cited for VNS from 2000 is accurate but potentially misleading. VNS was responsible for both exit polls and reporting final vote counts for every race (the latter function is now provided by the Associated Press). The costs also vary considerably between presidential and off-year elections.  Finally, the NEP exit operation still includes more than just exit polls, it also collects vote results at samples of key precincts and provides statistical modeling and analysis used to "call" races.

 

Comments
Geoff Dougherty:

Mark,

The final tally came out to 132 respondents at 12 polling places.

I've posted some details on the methodology here: http://www.chicagocurrent.com/articles/31085-Behind-the-scenes-details-on-the-exit-poll

I would agree with you that not all polls are created equal, and that ours is clearly not the gold standard.

However, it provided some accurate early intelligence on who was going to win the election, and some fascinating insight into why the victors prevailed.

That's really all we can ask for.

____________________

poughies:

And thank God the NEP no longer provides vote data... If I'm not mistaken (and someone can correct me), it was the VNS data (that was faulty due to a Volusia County machine error that was never corrected) that caused the networks to call the election for Bush... Meanwhile the APs count (and Sec of State) differed, and the AP never called the race...

As for the exit poll in Chi-town... it was a good try and hopefully something that will be tried in the future...

____________________

Thomas Riehle:

Geoff Dougherty seems like a very nice guy, and he chose an excellent advisor in Phil Meyer (although he concedes he didn't listen to him).

But as a pollster myself, I am not sure giving Geoff's adventure an A for effort is good for the industry. I know this looks easy, but kids, don't try this at home!

____________________

Geoff Dougherty:

@Thomas: I was being a bit flippant about not listening to Phil -- we implemented all the advice that we had the logistics capacity to handle.

Next time we'll have more capacity, and will be able to follow that plan to the letter.

Difficult to see how a poll that comes with a clear explanation of its limitations would be bad for anyone.

It was clearly something our readers found a lot of value in, and at the end of the day, that's what matters for us.

____________________

Nick10:


Yes you can try this at home.

I conducted exit polls on over 50 races for 25 years. In most of those years my client was Milwaukee’s WTMJ-TV. Before then it was for CBS2 in Chicago (where politics is local) including two upset wins for Mayor in1979 and 1983.

We went on the air with data not weighted by actual voting soon after the polls closed. When you have a single client a (i.e., single payer) budget was an issue and we wanted a time advantage over competition.

Our second of only two misses was in 2006, the WI AG race, one of three races we polled. I retired from exit polls after then. Democratic bias had begun to appear in recent years in Wisconsin, a special problem using unweighted poll results. As you know, Democratic bias had been a problem for national exit poll consortiums in multiple states - even after weighting by votes.

Our sample was polling places - not precincts - an advantage in Wisconsin because of September primaries. I found that polling place locations and precincts for the general elections are known. Our sample was 60 or more locations typically 3000 or more interviews. Keep in mind that cluster samples have larger margins of error. The only weighting we did was based on past turnout in the locations sample.

Need to know more? Go to the “market shares” to the right on this page. Click “Media Polls” then “View Exit Poll Summary”. The 2006 results are in a special analysis, not shown.

Nick Panagakis.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR