29% approve of the job George Bush is doing as president,
Among 324 registered Republicans and those who lean
Republican, former Mayor Rudy Giuliani leads former Sen.
Fred Thompson (30% to 22%) in a national primary; Sen. John
McCain and former Gov. Mitt Romney trails at 13%. All
other candidates receive less than 5% each.
Among 422 registered Democrats and those who lean
Democratic, Sen. Hillary Clinton leads Sen. Barack Obama
(44% to 23%) in a national primary; former Sen. John
Edwards trails at 14%. All other candidates receive less
than 5% each.
A new Greenberg Quinlan Rosner/Democracy Corps poll, taken 7/25-29/07 finds approval of President Bush at 36%, with disapproval at 60%. The new Pew Center poll, also taken 7/25-29/07 puts approval at 29%, disapproval at 61%.
With these two new polls the trend estimate of approval now stands at 30.8%.
The GQR/Democracy Corps poll is a statistical outlier, falling above the 95% confidence interval around the approval trend. At the moment we have two outliers-- ARG's 25% is below the confidence interval.
GQR/Democracy Corps polls are of likely voters, rather than adults as with most polls here. As a result, they consistently estimate approval levels higher than for the adult population and hence above the trend line as well. This is clear from the plot below showing "Greenberg" surveys tracking high. This is a "design decision" to survey likely voters, and should not be considered a defect in the poll. Inferences are to a different population from that of the general population.
The "house effect" estimate for GQR/Democracy Corps surveys is just below 3 percentage points, with a confidence interval from just under 2 to just under 4 points. Despite this substantial house effect, GQR surveys have only rarely exceeded the confidence interval for presidential approval.
As the number of recent polls has increased, the standard "blue line" estimator of support has increased it's "bend" and is coming closer to the more sensitive "red line" estimator, further evidence that approval trends changed direction around June 28, when the red estimator put approval at 28.7%. Currently the blue estimator agrees with that date as the turning point, though the estimate will continue to change until enough data are available for a stable estimate of the turning point.
While the change in support is now pretty clear, the reasons are less so. The timing coincides with a series of Supreme Court decisions, all carried by the majority created with Bush's appointments of Roberts and Alito. I don't think the Court is salient enough with the general population to be a strong driver of public opinion, especially of presidential approval. However, it is possible that the most politically involved conservatives were both aware of the decisions and gave Bush credit for his appointments, helping arrest his decline in the polls.
The other event about this time was the commutation of Scooter Libby's prison sentence on July 2. Initial reaction among the general population and even many self-described conservatives was disapproving of this decision. But here too the effect may have been positive among conservative Republicans and served to shore up support among that constituency.
But that said, unlike previous sharp turning points that have corresponded to major presidential speeches, this one is harder to account for with "obvious" actions of the President (or of Congress, for that matter.) I invite your speculation on this.
Two new statewide surveys from American Research Group of likely voters in California and Alabama (conducted 7/30 through 8/2) finds:
Among 600 Democrats in California, Sen. Hillary Clinton leads Sen. Barack Obama (35% to 22%); former Sen. John Edwards trails at 16%, Sen. Joe Biden and Gov. Bill Richardson both at 5%. Among 600 Democrats in Alabama, Clinton (at 38%) leads both Edwards (19%) and Obama (17%). All other candidates receive less than 5% each.
Among 600 Republicans in California, former Mayor Rudy Giuliani (at 30%) leads former Sen. Fred Thompson and former Gov. Mitt Romney (both at 18%); Sen. John McCain and former Speaker Newt Gingrich both trail at 7%. Among 600 Republicans in Alabama, Thompson leads Giuliani (31% to 26%); McCain trails at 16%, Gingrich at 8%. All other candidates receive less than 5% each.
View all California Primary poll data at Pollster.com here:
A new ABC News/Washington Post statewide survey (ABC story, results; Poststory, results) of 500 likely** Democratic caucus goers in Iowa (conducted 7/26 through 7/31) finds a "three-way tie" between Sen. Barack Obama (27%), Sen. Hillary Clinton (26%) and former Sen. John Edwards (26%); Gov Bill Richardson trails at 11%. All other candidates receive less than 5% each.
** Likely voters were chosen if the respondents said they were registered to vote in Iowa at their present address, if they were certain to attend or probably will attend the Iowa caucus, and if they would vote specifically in the Democratic Party caucus.
I want to pick up where I left
off on Tuesday, when I wrote about the way national surveys screen for
primary voters. How well have the pollsters in early primary states done in
disclosing how tightly they "screen" to identify the voters that will actually
turn out to vote (or caucus)? Not very well, unfortunately.
For those just dropping in, here is the basic dilemma: Voter
turnout in primary elections and, especially in caucus states like Iowa, is typically much
lower than in the general election. A pre-election survey that aims to track
and ultimately project the outcome of the "horse-race" -- the measure of voter
preferences "if the election were held today" -- needs to represent the population
voters." When the expected turnout is very low, that becomes a difficult
task, especially when polling many months before an election.
And in Iowa and South Carolina, if
history is a guide, that turnout will be a very small fraction of eligible
adults,** as the following table shows:
When a pollster uses a random digit telephone methodology,
they begin by randomly sampling adults in all households with landline telephone
service. They need to use some mechanism to identify a probable electorate from
within a sample of all adults. If recent history is a guide, the probable
electorate in Iowa
-- Democrats and Republicans -- will fall
in the high single digits as a percentage of eligible adults. South Carolina's turnout is better, but is
still unlikely to exceed 30% of adults. And while the New Hampshire primary typically draws the
highest turnout of any of the presidential primaries, it still attracts less
than half of the eligible adults in the state. Despite all the attention the New Hampshire primary
receives, many voters that ultimately cast ballots in the November general election
(roughly 30% in 2000) choose to skip their states' storied primary.
A pollster may not want to "screen" so that the size of
their likely voter matches the exact level of turnout. Most campaign pollsters
I have worked with prefer to shoot for a slightly more expansive universe, both
to capture those genuinely uncertain about whether they will vote and to
account for the presumption that "refusals" (those who hang up on their own
before answering any questions) are more likely to be non-voters.
Nonetheless, the degree to which pollsters screen matters a
great deal. If, hypothetically, one Democratic primary poll captures 10% of
eligible adults while another captures 40%, the results could easily be very
different (and I'll definitely put more faith in the first).
It also matters greatly how
the pollster go about identifying likely voters. I wrote quite a bit about
that process in October 2004 as it applies to random digit dial (RDD) surveys
of general election voters. In extremely low turnout contests, such as the Iowa caucuses, most
campaign pollsters now rely on samples drawn from lists of registered voters
that include the vote history of individual voters. Most of the Democratic
pollsters I know agree with Mark Mellman, who asserted in a must-read column
in The Hill earlier this year that,
"the only accurate way to poll the Iowa caucuses starts with the party's voter
So, based on the information they routinely release, what do
we know about way the recent polls in Iowa, New Hampshire and South
Carolina screened for likely voters? As the many
questions marks in the tables below show, not much.
The gold star for disclosure goes to the automated pollster
SurveyUSA. Of 22 survey organizations active so far in these states, they are
the only organization that routinely releases (and makes available on their web
site) all of the information necessary to determine how tightly they screen. Every
release includes a simple statement like the one from their May
poll of New Hampshire
2,000 state of New
Hampshire adults were interviewed by SurveyUSA
05/04/07 through 05/06/07. . . Of the 2,000 NH adults, 1,756 were registered to
vote. Of them, 551 were identified by SurveyUSA as likely to vote in the
Republican NH Primary, 589 were identified by SurveyUSA as likely to vote in
the Democratic NH Primary, and were included in this survey.
I did the simple math using the number above (which are weighted
values). For SurveyUSA's May survey, Democratic likely voters represented 29%
of adults and Republican likely voters represented 28%, for a total of 57% of
all New Hampshire
adults. Their screen is a very reasonable fit for a survey fielded eight months
before the primary.
Honorable mention for disclosure also goes to two Iowa polls. First, the Des MoinesRegister poll conducted by Selzer
and Company. Ann Selzer provided me with very complete information upon
request last year. Her first Iowa caucus
survey last year used a registered voter list sample and screened reach a
population that represents roughly 11% of the eligible adults (assuming 2.0
voters in Iowa
and 2.2 million eligible
Second, the poll conducted in March by the University of Iowa.
While their survey asked an open-ended vote question (rendering the results
incomparable with those included in our Iowa chart),
did at least provide the basic numbers concerning their likely voter screen. They
interviewed 298 Democratic likely caucus goers and 178 Republican caucus-goers
out of 1,290 "registered Iowa
voters" (for an incidence of 37% of registered voters). Unfortunately, they did
not specify whether they used a registered voter list or a random digit sample,
although given the incidence of registered voters in Iowa, we can assume that the percentage of
eligible adults that passed the screen was probably in the low 30s.
And speaking of the sampling frame, only 6 of 22
organizations SurveyUSA, Des Moines
Register/Selzer, Fox News, Rasmussen Reports, Zogby, and Winthrop
University specified the sampling method they used (random digit dial, RBS or
listed telephone directory). I will give honorable mention to two more
organizations -- Chernoff Newman/ MarketSearch and the partnership of Hamilton
Beattie (D) and Ayres McHenry (R) -- that disclosed their sample method to me
upon request earlier this year.
The obfuscation of this information by the remaining 14
pollsters is particularly stunning given that the ethical codes of both the
American Association for Public Opinion Research (AAPOR) and the National
Council on Public Polls (NCPP) include explicitly
require the disclosure of the sampling method, also known as the sample
"frame." The NCPP's principles of
disclosure requires the following for its member organizations for "all reports of survey findings issued for public release:"
Sampling method employed (for
example, random-digit dialed telephone sample, list-based telephone sample,
area probability sample, probability mail sample, other probability sample,
opt-in internet panel, non-probability convenience sample, use of any
definition of the population under study, and a description of the sampling
frame used to identify this population.
Finally, while virtually all of these surveys told us how
many "likely primary voters" they selected, very few provided details on how
they determined that voters (or caucus goers) were in fact "likely" to
participate. The most notable exceptions were the Hamilton
Beattie (D) Ayres McHenry (R) and Chernoff
Newman/ MarketSearch polls in South Carolina,
and the News
7/Suffolk University poll in New
Hampshire. All of these included the questions used
to screen for likely primary voters in the "filled-in" questionnaires that
included full results.
So what should an educated poll consumer do? I have one more
category of diagnostic questions to review, and then I want to propose
something we might be able to do about the very limited methodological
information available to us. For now, here's two-word hint of what I have in
mind: "upon request."
**Political scientists typically use two statistics to calculate turnout among adults: all adults of voting age (also known as
the voting age population or VAP), or all adults who are eligible to vote (or
the voter eligible population or VEP). George Mason University Professor
Michael McDonald has helped popularize
VEP as a better way to calculate voter turnout, because it excludes adults
ineligible for voting such as non-citizens and ineligible felons. The perfect
statistic for comparison to telephone surveys of adults would fall somewhere in
between, because adult telephone samples do not reach those living in
institutions or who do not speak English, but might still include non-citizens
that speak English (or Spanish where pollsters use bilingual interviewers).
In a state like California,
with a large non-citizen population, VAP is probably the better statistic for
comparisons to the way polls screen for likely voters. In Iowa,
New Hampshire and South Carolina, however, the choice has very
little impact. Had I used VAP rather than VEP above, the turnout statistics in
the table would have been roughly a half a percentage point lower.
CORRECTION: Due to an error in my spreadsheet, the original version of the turnout table above incorrectly displayed turnout as a percentage of VAP rather than VEP. For reference, the table below has turnout as a percentage of VAP.
A new Pew Research Center national survey (summary, Congress, SCOTUS, Iraq, 08; results) of 1,503 adults (conducted 7/25 through 7/29 by Schulman, Ronca & Bucuvalas, Inc.) finds:
29% approve of the job George Bush is doing as president; 61% disapprove.
Among 623 Democratic voters and those who lean Democratic, Sen. Hillary Clinton leads Sen. Barack Obama (40% to 21%) in a national primary -- widening her advantage to "nearly two-to-one;" former V.P. Al Gore trails at 12%, former Sen. John Edwards at 11%. All other candidates receive less than 5% each.
Among 546 Republican voters and those who lean Republican, former Mayor Rudy Giuliani (at 27%) leads former Sen. Fred Thompson (18%) and Sen. John McCain (16%) in a national primary; former Gov. Mitt Romney trails at 10%, former Speaker Newt Gingrich at 8%. All other candidates receive less than 5% each.
43% say the U.S. will "definitely" or "probably" succeed in achieving its goals in Iraq; 49% say it will "definitely" or "probably" fail.
New polls from Hotline and NBC/WSJ continue to raise the trend estimate for approval of President Bush, taking the estimate above 30% for the first time in a while.
Hotline, taken 7/19-22/07 has approval at 33%, disapproval at 63%. The NBC/Wall Street Journal poll done 7/27-30/07 has approval at 31 and disapproval at 63%.
With these new data, the approval trend now stands at 30.2%. This is in keeping with the upturn in approval first noted here, and confirmed by recent polls. Of course, 30.2% is nothing to brag about, but it is a tad better than being in the 20s.
The exact timing of the change remains unclear from my standard estimator, but the more sensitive "red" estimator thinks the turn was sharp and the rise has been substantial. Remember, "red" is often fooled by a few polls, so retain some skepticism about the red estimator, but we've got enough recent polls to make me pretty confident "red" has the story right, if not the exact timing or extent of the rise.
With the new trend estimate, the ARG poll at 25% has now moved clearly into outlier territory. This is a good example of how determining what is or is not an outlier can shift as more complete data arrive.
Yesterday, in response to this post, readers raised a number of excellent questions about the effects of individual polls in our trend estimates of candidate support (and just about everything else here as well, including presidential and congressional approval, support for the war and more.) Much of the discussion was about how to detect and exclude "bad" polls, which is a topic that covers a huge range of issues including "house effects" (the tendency of polling organizations to poll consistently high or low on some questions), outliers (single polls that are far from the rest) and more. The discussion will provide fodder for a number of posts to come later this month as I review our methods and try to clarify these and other issues. So there is a lot to do. Consider this a down-payment on the rest.
To paraphrase one question: "Why not exclude a polling organization if it consistently produces results out of line with everyone else?"
We could approach this in several ways. For example, suppose a pollster was consistently 4 percentage points high but their polls moved in synch with the trend in all the other polls. Movement in those polls would tell you a lot about dynamics of opinion even if the pollster were "biased" by 4 points. If the bias were consistent, then we could just subtract 4 points and have an excellent estimate of the trend. A simple shift of the average poll result above or below the overall trend is not in and of itself a clinching argument for excluding a pollster. I'll come back to this issue in much more detail later in this series of posts.
A simpler and more direct way to approach the question is to ask what difference does it make if we do include all polls, rather than exclude supposedly "bad" ones? Of course we'd have a major problem if the trend estimator were quite sensitive to individual polls or all the polls by a particular pollster. Happily, this is an empirical question, so we can answer it. And we don't have to know which pollster is "bad" to begin with.
The plots above show the trend estimate, using our standard estimator, as the black line. This uses all the polls we have available for the national nomination contests in both parties. The light blue lines are trend estimates that result when I drop each of the 19 different polling organizations, one at a time. Though the lines are indistinguishable, there are 19 different blue ones for each candidate in the figures. If the impact of individual organizations on the trend estimate were large, some of these blue lines would diverge sharply from the black overall trend line and we'd be seriously concerned about those polls that were responsible for the divergent results.
But that isn't what actually happens. The blue lines all fall within +/- 1 percentage point of the overall trend estimate and the vast majority are within less than +/- 0.5 points. There is no evidence that excluding any single organization has more than a trivial effect on the estimated trend. This alone is strong evidence that whatever problems specific pollsters or individual polls may have, they do not seriously disturb the trend estimates we use here at Political Arithmetik and Pollster.com.
It is interesting that the variation around the top candidates in both parties, Clinton and Giuliani, is larger than it is among the third place candidates, Edwards and Romney, while variation for the middle candidates falls in between. This is a possible clue to one aspect of "house effects". One well known source of house effects is due to how hard the interviewer pushes for an answer. Some organizations now routinely find 20% or more unable or unwilling to pick a candidate. Other organizations have less than 5% failing to choose a candidate. Now imagine yourself asked to pick, but lacking an actual preference. When pushed, who do you most likely "settle" for in order to placate the interviewer? I'd bet on the best known names. If that were the case, we'd see the greater variation around Clinton and Giuliani substantially explained by differences in how hard pollsters push for answers on the vote preference question. This is one more topic for another day.
The fact that we find little effect on the trend estimate due to excluding each pollster could mean one or both of two things: either no pollster is biased or discrepant enough to actually raise a problem in the first place, or the trend estimator we are using is statistically robust enough that it resists the influence of unusual pollsters or polls. The second possibility is true by design. I've chosen an estimation method and designed the approach we take so that the trend estimator should be resistant to bias due to a single organization or a single poll. While it can be fooled under the right circumstances, those should be both rare and short lived, rather than common and long term.
We are not in a position today to reach a conclusion about the first possibility, that none of our pollsters are consistently out of line with the others. That could be, but it could also be that one or more pollsters are in fact out of step but that the estimator successfully resists their influence. To address this more interesting question, will require more work and a separate post (or series of posts). It does seem to me that there are clearly systematic differences across polling organizations. I've done a good many posts in the past on "house effects" and on individual outliers, and will do more of that in the coming weeks. But it also disturbs me that many complaints are hurled at specific polling organizations with little or no effort to support the claims empirically and systematically. That is a job we'll begin to undertake here as a route to clarifying what the actual evidence is for which polls are less "reliable" than others, and what exactly that means. Stay tuned.
New analysis from recent Gallup national surveys of adults that during the second quarter of 2007, finds 37% of adults call themselves independents, "a small -- but significant -- increase" since 2006.
Among 839 likely voters nationwide, both former Mayor Rudy Giuliani and former Sen. Fred Thompson edge out Sen. Hillary Clinton (46% to 45% each) in general election match-ups (conducted 7/30 through 7/31).
Among 1,200 likely voters nationwide, Rep. Dennis Kucinich trails both Giuliani (by 14 points) and Thompson (by 9 points) in general election match-ups.
Additional results from the new NBC News/Wall Street Journal national survey (NBC results; WSJstory, results) of 1,005 adults (conducted 7/27 through 7/30) finds:
31% approve of the job George Bush is doing as president; 63% disapprove.
68% either believe the country "is currently in an economic recession" or "will be within the next twelve months;" 25% believe the country is not in a recession and will not be in the next twelve months.
Among likely Republican primary voters nationwide, former Mayor Rudy Giuliani (at 33%) leads former Sen. Fred Thompson (20%) and Sen. John McCain (17%) in a statewide general election match-up; former Gov. Mitt Romney trails at 11%. All other candidates receive less than 5% each.
Republican pollster David Hill ponders the difficulty of
polling in primaries and the August "Twilight
Zone of presidential politics in which pollsters will have a hard time
determining what's real and what is illusion in poll results."
The first round of results from the new NBC News/Wall Street Journal national survey (WSJstory, results) of 1,005 adults (conducted 7/27 through 7/30) finds:
Among Democrats, Sen. Hillary Clinton leads Sen. Barack Obama 43% to 22% in a national primary; former Sen. John Edwards trails at 13%, Gov. Bill Richardson at 6%, Sen. Joe Biden at 5%. All other candidates receive less than 5% each.
General election match-ups
Clinton 47%, Giuliani 41%
Clinton 42%, Giuliani 34%, Bloomberg 11%
Obama 45%, Giuliani 40%
25% think Clinton is "very liberal" in her approach to the issues; 21% say "somewhat liberal," 35% say "moderate," 7% say "somewhat conservative," and 3% say "very conservative."
A new Siena College statewide survey of registered voters in New York State (conducted 7/24 through 7/28) finds:
Among 176 Republicans asked to choose between five candidates, former Mayor Rudy Giuliani (at 40%) leads Sen. John McCain (13%), former Sen. Fred Thompson (11%), former Speaker Newt Gingrich (9%), and former Gov. Mitt Romney (7%) in a statewide primary.
Among 290 Democrats asked to choose between eight candidates, Sen. Hillary Clinton (at 48%) leads Sen. Barack Obama (14%), former V.P. Al Gore (10%), and former Sen. John Edwards (7%). All other candidates receive less that 5% each.
General election match-ups:
Clinton 57%, Giuliani 36%
Clinton 44%, Giuliani 26%, Bloomberg 22%
Clinton 62%, Thompson 28%
Obama 51%, Giuliani 40%
Obama 56%, Thompson 25%
View all New York Primary poll data at Pollster.com:
Former NBC News political director Elizabeth Wilner devotes
her Politico column to the state of
media polling. She leads with this unhappy speculation about the NBC/Wall
Hanging in the balance of Rupert
Murdoch's courtship of Dow Jones is the fate of one of the stalwart franchises
in public opinion research on American politics and policy.
Competition being what it is, it's hard to imagine that a Murdoch-owned Wall Street Journal would
be allowed to continue its nearly 20-year polling partnership with NBC News.
Particularly when that partnership benefits business news channel CNBC, which
Murdoch hopes to crush with a new, Journal-powered Fox business network.
The prospect of what Wilner later describes as a potential "NBC/Journal
divorce" is a sad one for those of us who value the best in political
surveys. Here is hoping that Murdoch lets this partnership continue.
Wilner has more on the way the news media covers polls. Read it all.
Update (Dept. of Unintended Irony): As it happens, the latest NBC/WSJ poll is out this morning (sub. req.), and shows Hillary Clinton widening her lead among Democrats, leading Barack Obama 43% to 22%, with John Edwards in third and other Democrats "in single digits." More details later.
The release of new
surveys yesterday by the American
Research Group (ARG) in Iowa, New Hampshire and South Carolina has
generated quite a bit of buzz as well as questions about ARG of the sort I
alluded to yesterday in my post on the screens pollsters use to select likely primary
voters and caucus attendees. That post is the first in a series that will look
at the methodological differences among the various pollsters active in the
early primary states. For now, however, let me take up two issues specific to Iowa and ARG's surveys
The first question comes from reader AL:
think your averaging is wrong on the Iowa
race for the democrats. I think it should be Edwards at 25.7 and Hillary at
25.4. You have the averaging mixed up. You should correct it.
I am not sure how AL arrived
at those numbers, but the current Pollster.com estimates in Iowa (Clinton 25.7%,
Edwards 25.4%) are not "mixed up." The confusion may arise from the fact that
our estimates are regression based estimators rather than true averages. Professor
Franklin explains the difference in detail here.
The second issue concerns the Iowa
surveys by ARG.
As commenter jsamuel put
it, "ARG seems to almost always under poll Edwards." While I lack Professor
Franklin's flair for graphics and regression trend lines, some simple averages show
that jsamuel is right. Sen. Clinton does consistently better ARG's surveys than
those from other pollsters, while former Sen. Edwards does consistently worse:
We logged in six new Iowa
surveys during June and July. The two from ARG show Clinton ahead of Edwards by an average six
points (31% to 25%), while the four from other pollsters give Edwards an
average lead of five points (27% to 22%).
Surveys from February, March, April and May show essentially the same
leads by two points (30% to 28%) in four surveys conducted by ARG, while
Edwards leads by an average six points (27% to 21%) in 13 surveys by other
Unfortunately, this is the sort of scenario in which
averaging (or simple regression based estimation) can be potentially
misleading. One pollster (ARG) is getting consistently different results and contributing a large number of polls
to our overall estimate. So if ARG is both different and wrong, their polls are throwing off our estimates.
We know ARG's results in Iowa are different. Why? And what should we
make of that difference? For some clues, stay tuned to my series on primary
As everyone knows, the presidential contests started early this round. But how much attention are the races getting from ordinary people (i.e. not us junkies here.) The debates have had small audiences, but news coverage is quite substantial. How much attention are voters paying?
The Pew Center has asked a consistent question on attention to the campaign going back to 1988. Their question:
As I read a list of some stories covered by news organizations this past week, tell me if you happened to follow each news story very closely, fairly closely, not too closely, or not at all closely. News about candidates for the 2008 presidential election?
Looking at those saying they are following the news about the campaign "very closely" we find the pattern in the figure above. Around 20% of the sample of adults say they are paying this much attention to news about the election. On the one hand, that means many are not. On the other, no previous election for which we have polling this early comes close to that 20% figure. The three previous races with any polling as early as this found 10-15% paying "very" close attention to the campaign. Only about 10 months before election day, or about the start of the primaries and caucuses, do we typically see interest move up to high levels, varying from about 18-35% over the various election years.
So this year's level of interest looks quite high, compared to previous years about this time, but at an absolute level, it remains only a fifth of adults who are glued to campaign news.
If we add "very closely" with "fairly closely", the current interest is 53%, a substantial increase, but the pattern in the plot is unchanged-- 2007 stands above any previous year, but the final attention by the general election rises to over 90% when using this combined measure, so 53% now is certain to rise to near 100% by November 2008. So either analysis says current attention is well above the norm, but still has a lot of room for increase.
The most interesting races have, oddly enough, involved incumbents named Bush. The 1992 three way race and the 2004 contest provoked the greatest attention from the Pew samples. Curiously the closest of all contests, 2000, had the second lowest final attention measure of the five completed races in the plot.
Comparing attention today with past races, we appear about 5-6 months ahead of attention in previous cycles. The 20-25% paying high attention now is usually reached around January of the election year. One interesting question is how much, and how soon, attention will grow. Perhaps we are starting high this cycle, but will converge to the range of previous cycles by January. Or perhaps we'll remain above past races through November 2008. Wait and see.
The one constituency that is certainly paying more attention earlier is pollsters. At this point in the 2004 cycle, there had been 53 national nomination polls. As of July 22 this year, we had 109, more than double the previous pace.
In part this is because we started earlier this year:
The earliest consistent polling on the 2004 race began after November 2002. (I may have missed a handful of earlier polling, but certainly not a great many.) In contrast, regular polling on 2008 began immediately after November 2004. The density of polls in 2005 and especially 2006 was substantially above comparable levels in 2002-03.
However, once the contest ramped up, the rate of new polls has been quite similar between the two years. When the blue line for 2008 turns sharply up, it runs in parallel to the 2004 polling. Clearly 2008 will ultimately far outstrip 2004, but for now we are seeing comparable rates of polling, just shifted back in time.
The shift is the equivalent of six months in the campaign schedule. If I shift the 2004 data back by exactly 6 months, the result is a near perfect match, seen below (again, after the longer run up in 2004-2006).
So for those of us obsessed with campaigns and public opinion, this is great news-- the most studied race in history in going on right now, and we have box seats.
The questions we seem to get most often here at Pollster,
either in the comments or via email, concern the variability we see in the
presidential primary polls, especially in the early primary states. Why is
pollster A showing a result that seems consistently different than what pollster
B shows? Why do the results from pollster C seem so volatile? Which results
should we trust? I took up one such conflict last Friday.
Unfortunately, definitive answers to some of these questions
are elusive, given the vagaries of the art of pre-election polling in
relatively low turnout primaries. When confronted with such questions,
political insiders tend to rely on conventional wisdom and pollster reputation. Our preference
is to look at differences in how
survey results were obtained and take those differences into account in
analyzing the data.
At various AAPOR conferences in recent years, I have heard the
most experienced pollsters repeatedly confirm my own intuition: To find the
most trustworthy primary election polls, we need to look close at how tightly
the pollsters "screen" for likely primary voters. In other words, primary and
caucus turnout is usually low in comparison to general elections. In 2004 (by
my calculations), the Democratic turnout amounted to 6% of the voting age
population for the Iowa Caucuses and 22% for the New Hampshire primary. In other states,
turnout averaged 9% in primary states and 1.4% in caucus states in 2004.
A pollster that begins with a sample of adults has to narrow
the sample down to something resembling the likely electorate, which is not
easy. As few will approach the task exactly the same way, this is an area of
polling methodology that is much more about art than science. Nonetheless, in
most primary polls, relatively tighter screens are preferable in trying to
model a likely electorate.
Thus, to try to make sense of the polls before us we want to
know two things. First, how narrowly
did the pollsters screen for primary voters? Second, as no two such screens are
created equal, what kind of people
qualified as primary voters?
In this post, I will look at what some recent national polls
have told us about how tightly they screened their samples before asking a
presidential primary trial-heat question and what kinds of voters were
selected. I will turn to statewide polls in Part II. The table below summarizes
the available data, including the percentage of adults that get the Democratic
or Republican primary vote questions (if you click on the table, you will get a
pop-up version that includes the sample sizes for each survey).
Unfortunately, of the 20 national surveys checked above,
only five (Gallup/USA Today,
AP-IPSOS, CBS/New York Times, Cook/RT
Strategies and NBC/Wall Street Journal)
provide all of the information necessary to quantify the tightness of their screen
question. Others fall short. Here is a brief explanation at how I arrived at
the numbers above.
The calculation is easiest when the pollster reports results
for a random sample of all adults as well as the weighted value the subgroups
that answered the primary vote questions. In various ways, these five
organizations included the necessary information in readily available public
Five more organizations (CNN/ORC, Newsweek, LA Times/Bloomberg, the Pew
and Time) routinely provide the
subgroup sizes for respondents that answer primary vote questions, though they
do not specify whether the "n-sizes" are weighted or unweighted. Pollsters typically
provide unweighted counts because they are most appropriate for calculating
sampling error. However, since the unweighted statistic can provide a slightly
misleading estimate of the narrowness of the screen, I have labeled the percentages
for these organizations as approximate.
Of those that report results among all adults, only the ABC
News/Washington Post poll routinely
omits information about the size of the subgroups that answer primary vote
questions. Even though their articles and reports oftenleadwithresults
among partisans, they have provided no information about the sub-group sizes or
margin of error for party subgroups since February. While the Washington Post provided results for
party identification during 2005
and 2006, that practice appears to have ended changed as of February
[CORRECTION: The June and July filled-in questionnaires available at washingtonpost.com include the party identification question, and those tables also present time series data for the February and April surveys. However, as these releases do not include the follow-up question showing the percentage that lean to either party (which had been included in Post releases during 2006), they still do not provide information sufficient to determine the size of the subgroups that answered presidential primary trial-heat questions].
Determining the tightness of the screen gets much harder
when pollsters report overall results on their main sample for only registered
or "likely" voters. Three more organizations (Diageo/Hotline, Fox News/Opinion
Dynamics and Quinnipiac) provide overall results only for those who say they
are registered to vote. For these three (denoted with a double asterisk in the
table), I have calculated an estimate of the screen based on the educated guess
that roughly 85% of adults typically identify themselves as registered voters
on other surveys of adults.
Four more organizations (Rasmussen Reports, Zogby, and
Democracy Corps and McLaughlin and Associates) report primary results as
subgroups of samples of "likely voters." Since their standard releases provide
no information on how narrowly they screen to select "likely voters," we have
no way to estimate the tightness of their primary screens. If we simply divided
the size of the subgroup by the total sample, we would overstate the size of
the primary voting groups in comparison to the other surveys.
Finally, the American Research Group follows a procedure
followed for many statewide surveys: It provides only the number of interviews
asked the primary vote question with no information about the size of the
universe called to select those respondents.
All of the discussion above concerns the first question: How narrowly did the pollsters screen? We
have somewhat better information -- at least with regards to national surveys --
about the second question: how those
people were selected. The last column in the table categorizes each
pollster the by the way they select respondents to receive primary vote
Partisans -- This is the approach taken by Gallup/USA Today, ABC
News/Washington Post, AP-IPSOS. It includes, for each party, all adults
that identify or "lean" to that party.
Partisan+ -- The approach taken by NBC/Wall Street Journal includes both
party identifiers and leaners and those who say they typically vote in the
primary election of the given party. The LA Times/Bloomberg poll takes a
similar approach although its screen appears to exclude leaners.
Partisan or RV/Partisan -- This approach is taken by a large number of
pollsters. It takes only those partisans or "leaned" partisans that say
they are also registered to vote. Those labeled RV/Partisan exclude party
"leaners" from the subgroup.
Voters -- This category includes the surveys that use questions about
primary voting (rather than party identification) to select respondents that
will be asked primary vote questions.
As should be apparent from the table, the pollsters that use
the "leaned partisan" or "leaned partisan+" select partisans more broadly than
those that include only registered voters or those that claim to vote in
primaries. But all of these approaches are getting a much broader slice of the
electorate than is likely to actually participate in a primary or caucus in
2008. As should be obvious, most of the national pollsters are not trying to
model a specific electorate -- they are mostly providing data on the
preferences of "Democrats" or "Republicans" (or Democratic or Republican "voters").
I wrote about that issue and its consequences back in
In Part II, I will turn to statewide polls in the early
primary states and then discuss what to make of it all. Unfortunately, while
the information discussed above is incomplete, the national polls look like a model of
disclosure as compared to what we know about most of the statewide polls.
A new Gallup panel survey of 1,011 adults nationwide (conducted 7/23 through 7/26) finds:
55% have "a great deal" or "a fair amount" of confidence in Sen. John McCain "to do or to recommend the right thing for the war in Iraq;" 55% for former Mayor Rudy Giuliani, 54% for Barack Obama, 51% for Sen. Hillary Clinton, and 50% for Sen. John Edwards.
69% have a great deal or a fair amount of confidence in Giuliani "to do or to recommend the right thing for handling terrorism;" 66% for McCain, 55% for Clinton, 53% for Obama, and 48% for Edwards.
62% have a great deal or a fair amount of confidence in Obama "to do or to recommend the right thing for the economy;" 61% for Clinton, 60% for Giuliani, 53% for MCain, 51% for Edwards.
Three new American Research Group statewide surveys of 600 likely Democratic primary voters and 600 likely Republican primary voters in Iowa, New Hampshire, and South Carolina (conducted 7/26 through 7/30) finds:
Among Democrats in Iowa, Sen. Hillary Clinton leads former Sen. John Edwards (30% to 21%); Sen. Barack Obama trails at 15%, Gov. Bill Richardson at 13%. Among Republicans, former Mayor Rudy Giuliani edges out former Gov. Mitt Romney (22% to 21%); Sen. John McCain follows at 17%, former Sen. Fred Thompson at 13%.
Among Democrats in New Hampshire, Clinton and Obama are tied at 31%; Edwards trails at 14%, Richardson at 7%. Among Republicans, Giuliani edges out Romney (27% to 26%); Thompson trails at 13%, McCain at 10%, former Speaker Newt Gingrich at 6%.
Among Democrats in South Carolina, Obama edges out Clinton (33% to 29%); Edwards trails at 18%. Among Republicans, Giuliani edges out thompson (28% to 27%); McCain trails at 10%; Gingrich and Romney both at 7%.
Two new SurveyUSA automated surveys in Minnesota find:
Among 628 registered voters, Republican Sen. Norm Coleman leads Democratic challengers Al Franken (49% to 42%), Mike Ciresi (48% to 42%), and Jim Cohen (49% to 37%) in statewide senatorial general election match-ups (conducted 7/26 through 7/29).
In a second survey among 600 adults, 43% approve of the job Coleman is doing, 48% disapprove (conducted 7/13 through 7/15).
A quick reminder not to assume what is today will be so tomorrow.
The 2004 Democratic primary race as of late July 2003 showed a continuous first place held by Lieberman, though with a slow but steady erosion. Kerry and Gephardt locked in a long running tie, and Howard Dean a rising 4th place at about 12% support. Clark's late entry and sharp rise hadn't happened. Edwards looked like a goner as his initial 9% had sunk to about 5%. So from this, who would be the candidates left standing after Iowa?
But of course the dynamics changed. Between summer and late fall, Dean became the "inevitable" nominee, sparking talk of running mates and gaining Gore's December endorsement. Kerry by that point was under 10%.
And then Iowa happened, and support shifted dramatically to Kerry and Edwards and away from Dean. (And Gephardt and Lieberman were gone.)
To those who say this is clear evidence that early polling is useless, I'd say no-- early polling is reflecting the dynamics of the race. But the dynamics are highly fluid and the point of the polling is not to predict the winner from today's polls, but to understand how the race is moving and ultimately to look back at how we got to the final outcome.
The great mistake analysts make is to look at current polls and conclude from them that the dynamics are fixed. That Dean can't rise. That Dean is a lock. That Kerry was inevitable after all.
The current Democratic race appears, as of last week's polling, to be relatively static. And compared to the Republicans that is certainly true. But let's not jump to the conclusion that the polls after Labor Day have to look like today's just because today's look like June 1. The polls are of interest for what they show about the history of the race so far and how it stands today. Not for their ability to predict what happens in a month or two.
Among 643 likely voters, former Sen. John Edwards leads former Mayor Rudy Giuliani (49% to 42%) and leads former Sen. Fred Thompson (50% to 39%) in nationwide general election match-ups (conducted 7/25 through 7/26).
Among 1,472 likely voters, Sen. Barack Obama leads former Mayor Rudy Giuliani (47% to 41%) and leads former Sen. Fred Thompson (46% to 40%) in nationwide general election match-ups (conducted 7/23 through 7/24).