Pollster.com

Articles and Analysis

 

More on ARG and Iowa

Topics: 2008 , Disclosure , Likely Voters , The 2008 Race

Following up on yesterday's post, in which I speculated - wrongly, as it turns out -- about the incidence of eligible adults selected by the American Research Group (ARG) as likely caucus goers for their most recent surveys of Democrats and Republicans in Iowa. I emailed Dick Bennett, and can now report on how their surveys compare to the others that have provided us with similar details.

First, according to Bennett, I was incorrect in speculating that they use only one question to screen for "likely caucus goers." They start with a random digit dial (RDD) sample of adults in Iowa in households with a working telephone and then ask four different questions (although they provide only the last question on the page reporting Iowa results):

  • They ask whether respondents are registered to vote, and whether they are registered as Democrats or Republicans. Non-registrants are terminated and not interviewed.
  • They ask registrants how likely they are to participate in the Caucus "a 1-to-10 scale with 1 meaning definitely not participating and 10 meaning definitely participating." Those who answer 1 through 6 are terminated and not interviewed.
  • They ask unaffiliated registrants ("independents" registered as neither Democrats nor Republicans) whether they plan to participate in the Democratic or Republican caucus. Registered Democrats and independents who plan to caucus with the Democrats get the Democratic vote question; Registered Republicans and independents who plan to caucus with the Republicans answer the Republican question.
  • After asking vote question, they asks the question that appears on the web site: "Would you say that you definitely plan to participate in the 2008 Democratic presidential caucus, that you might participate in the 2008 Democratic presidential caucus, or that you will probably not participate in the 2008 Democratic presidential caucus?" Only the definite are included in the final sample of likely caucus voters.

So the process involves calling a random sample of adults until they reach a quota of 600 interviews for voters of one of the parties. In their most recent Iowa survey, they were able to fill the quota for Democrats first, so they continued dialing the random sample until they had interviewed 600 Republicans, terminating 155 Democrats in the process. Bennett reports that they also terminated another 4,842 adults on their various screen questions (740 who say they were not registered to vote, 3,598 who rated their likelihood of participating as 6 or lower and 504 who were less than "definite" about participating on the final question).

So, the "back of the envelope" calculation for ARG is that their most recent sample of Democrats represents 12% of Iowa adults (755 Democrats divided by 755+600+4,842). Their most recent sample of Republicans represents roughly 10% of Iowa adults (600 Republicans divided by 755+600+4,842). We can compare the Democratic statistic to those provided by other Iowa pollsters:

And again, for those just joining this discussion, the 2004 Democratic caucus turnout was reported as 122,200, which represented 5.4% of the voting age population and 5.6 of eligible adults.

So, if we take all of these pollsters at their word, my "blogger speculation" yesterday was off-base: ARG's incidence of Democratic likely voters as a percentage of eligible adults is very close to the surveys done by Time and ABC/Washington Post. Apologies to Bennett.

But we still have a mystery. Why the consistent difference between the result from ARG and other surveys that appears to favor Clinton? Professor Franklin is working on a post as I speak that will chart the difference, but when we exclude the ARG's surveys from our estimate for Iowa, Clinton's current 2 point margin over Edwards (26.2% to 24.2%) becomes a 1.3 point deficit (24.6% to 25.9%). [See Franklin's in-depth discussion, now posted here].

IAIATopDems-sml.png

I asked Bennett whether he had any theories that might explain the difference. Here is his response:

Our sample size is larger and our likely voter screen is more difficult to pass. As you have pointed out, many surveys (although they are not designed to project participation) project unrealistic levels of participation. A likely voter/participant does not need to vote/participate to represent the pool of likely voters/participants, but the likely voter/participant pool is not much larger than the actual turnout.

Our results in Iowa show that John Edwards has a slight lead over Hillary Clinton among those voters saying they have attended a caucus in the past. Hillary Clinton has a greater lead among those saying this will be their first caucus. Hillary Clinton also has very strong support among women who say they usually do not vote/participate in primary/caucus races - this is true in Iowa and the other early states

Sample size is largely irrelevant to the pattern in our chart. Smaller samples would explain greater variability, but not a consistent difference across a large number of samples. The observation in his second paragraph is much more important. Since ARG's previous releases did not mention these results, I asked for the question about past caucus participation and the associated results. His response:

The question is: Will this be the first Democratic caucus you have attended, or have you attended a Democratic caucus in the past?

We first asked this in Feb:

Feb - 41% first, 59% past
Mar - 44% first, 55% past
Apr - 39% first, 60% past
May - 45% first, 55% past
Jun - 42% first, 57% past
Jul - 40% first, 60% past
Aug - 43% first, 57% past

We can compare this result to similar questions or reports from other recent surveys and they show a clear pattern. The differences among the four pollsters are huge and show a clear pattern, consistent with the differences Bennett reports in his own surveys: John Edwards does better against Clinton as the percentage of past caucus goers increases.

08-31%20first%20time%20caucus%20goer.png

So what is the right number of past caucus goers? Bennett can certainly argue that the entrance polls from the 2000 and 2004 Caucuses are on his side. Bennett used exactly the same question as the network entrance poll, which reported the percentage of first-time Democratic caucus goers as 53% in 2004 and 47% in 2000. Of course, as we learned three years ago, exit polls have their own problems, and I am guessing that other pollsters will debate what past-caucus goer number is correct. We will pursue this point further.

Finally, it is worth saying that this exchange and my arguably unfair "blogger speculation" yesterday makes one thing clear: If we are going to dig deeper into these issues, we have an obligation to ask these questions (about incidence and sample characteristics) about all polls, not just those from ARG, Time and a handful of others.

Stay tuned.

 

Comments
hwc:

Thanks for the additional research.

The prior-caucus goer shift makes perfect sense. After all, the entire universe of 2004 caucus voters has roughly a 30% chance of having already caucused for and supported John Edwards.

Obviously, that Edwards support is real. But, as the sample approaches 100% of past caucus goers, the sample is going to trend more and more towards including Edwards voters.

This is probably a flaw in a sample, since prior caucusing doesn't seem to be a particularly strong predictor of current year caucus participation.

Probably the better predictor of caucusing might be a question asking "do you really, really strongly support your candidate".

That's why we are getting conflicting results. Clinton has the most "strong" support. Edwards has the most "prior caucus" support.

____________________

hwc:

Hillary Clinton also has very strong support among women who say they usually do not vote/participate in primary/caucus races - this is true in Iowa and the other early states

This should be a cautionary note to all pollster attempting to use tight turnout screens in the polling for this election. Any pollster who misses on projecting women's turnout is likely to miss on their polling. For example, adjust the data to reflect 2004 female turnout and you could miss by a mile if female turnout increases in the 2008 cycle.

I don't know how pollsters or poll watchers can get a handle on this, since there has never been a serious female Presidential candidate in 230 years. However, it's an issue that we all need to pay attention to.

Your ARG contact is basically waving this red flag in his comments about what he is seeing in his Iowa polling.

Clinton commented on this issue on Letterman. That she is seeing so many mothers bringing their daughters to meet her and so many 90+ year old women who come to tell her they were born before women could vote and want to live long enough to see a female President. Anne Korblut of the Washington Post talked of this phenomenom in a video piece she did at the Iowa state fair.

____________________

zogby:

Hello,

Fascination stuff. However, you seem to ignore a recent Zogby poll which also shows a big Clinton lead. Can you also shed some light on that poll?

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR