October 31, 2007

Disclosure Project: Which Iowa Pollsters Responded?

Five weeks ago, I kicked off our Disclosure Project by sending email messages to the pollsters that have released surveys in Iowa this year. I asked pollsters to identify their sample frame and their procedure for selecting likely voters (or in this case, their likely caucus goers) and to report both the demographics of their likely caucus-goer samples and the percentage of Iowa adults that likely caucus-goers represent. A fair number of pollsters replied within days, others ignored my requests while a handful requested more time or reported that they were working to provide answers. Now that five weeks have gone by, a report our progress is more than overdue.

Let me start today by reviewing which pollsters have worked to comply, and which have not. All together, we sent requests to the seventeen pollsters that have released Iowa surveys listed below.

The pollsters divide into four categories. The first are the media pollsters that are members of the American Association for Public Opinion Research (AAPOR), plus the academics at the University of Iowa that put out this week's "Hawkeye" poll. As the AAPOR Code of Professional Ethics requires disclosure of this sort of information on request, it is gratifying that most of the AAPOR members on the list cooperated fully within a few days of our request.** One partial exception is Newsweek. Although they answered all other questions, they have so far refused multiple requests to provide demographic profile data.

This week's release by the University of Iowa's Hawkeye poll included a detailed methodology statement that answered all questions but did not provide a demographic profile. We have requested the latter, although obviously the Univ. of Iowa pollsters have had far less time to respond than the others on the list.

Another category includes the two "automated" pollsters - those that conduct surveys using the "interactive voice response" IVR methodology that has respondents answer questions by pressing keys on their touch-tone phones. The two that have been active in Iowa - Rasmussen Reports and Public Policy Polling (PPP) - have both provided complete responses. We were especially pleased to see Rasmussen Reports provide a complete methodology page for their October survey that walked through answers to all of our questions, making the need for a formal request unnecessary.

Among other organizations, unfortunately, cooperation has been more disappointing. I am not particularly surprised at the less than complete disclosure from the partisan pollsters. The two campaign pollsters that had released internal surveys campaign surveys - Paul Maslin for the Richardson campaign and Jan van Lohuizen for the Romney campaign - responded via email and were willing to describe their sampling procedures generally and provide the text of their trial heat questions. However, both politely refused to provide demographic profiles, the specifics of how they defined likely caucus-goers or the percentage of adults those samples represent. On the other hand, the two pollsters for the One Campaign poll - Democrat Geoff Garin and Republican John McLaughlin - did provide complete responses for their Iowa surveys.

Perhaps I have a bias (as a former campaign pollster and long ago employee of Paul Maslin), but I am sympathetic to the reluctance of campaign pollsters to share these details. The sample design of an internal campaign poll will provide hints of a campaign's strategy - in this case what sort of atypical caucus goers the campaign may be targeting. Moreover, campaigns know that their releases will be met with skepticism, that many will dismiss their findings as propaganda regardless of what they choose to disclose. If they refuse to disclose their methodology, we are free to question or ignore their results. That is the chance they take.

But that rationale is entirely absent for the other independent "public" polling organizations on our list that have still not responded after five weeks time. To be fair, the American Research Group (ARG) did answer our questions about one of its polls in August (just before we kicked off the formal Disclosure project). ARG's Dick Bennett also emailed two weeks ago to say they planned to add incidence data and more demographic profile data to their online reports. Strategic Vision's David Johnson also emailed ten days ago to say he is "working on" providing a response. I sincerely hope to hear from both organizations soon, as we all are quite busy, and late is better than never. However, we have had no for-the-record response - and in most cases, no response at all -- from Insider Advantage, Research2000, Mason Dixon or Zogby International.

Unfortunately, the pollsters in this last group, the ones that have so far not disclosed the information we requested, have conducted the majority of polls released in Iowa so far this year. The table below, which adds two columns to the one above, shows that we lack disclosure for three quarters of the Iowa conducted this year.

What is particularly disappointing is that organizations like Mason-Dixon, Research2000 and Zogby International earn their revenue through contracts with news organizations. When politicians stonewall and refuse requests for greater transparency, reporters howl. But when the news organizations that hire these pollsters conduct surveys, they allow their vendors to use the same rationale to stonewall disclosure as partisan campaign pollsters. Does that make any sense?

In my next post on this topic, I will review the answers we have received and talk about where we go from here.

**My own interests disclosed: I currently serve on AAPOR's Executive Council.

-- Mark Blumenthal

October 31, 2007 in Iowa

TrackBack

TrackBack URL for this entry:
http://www.pollster.com/cgi-bin/mt/mt-tb.fcgi/4333.

Listed below are links to weblogs that reference Disclosure Project: Which Iowa Pollsters Responded?:

Comments

Post a comment

URL:

Comments: (you may use HTML tags for style)

Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button.

Please email us to report offensive comments.

See our comment policy here. Note that we require commenters to share their email address via Typekey. We will never share your email address with anyone without your explicit permission.