A suggestion from alert reader and frequent commenter
I write to suggest that you analyze
the huge discrepancy between the latest Rasmussen and Washington Post/ABC
polls. I'm talking about the Republican nomination. Rasmussen says Thompson is
up by 4 over RG, while WP/ABC says Rudy is up by 20 pts over FT, who isn't even
in second place here (36 RG to 14 FT). One of these pollsters is
obviously very wrong. Two polls cannot both be accurate, if their margin of
victory do not approximate each other. This is a humongous 24 point
Here, with a little assist from Professor Franklin, is a
chart showing the discrepancy that Andrew noticed. The two surveys do seem to
show a consistent difference that is clearly about more than random sampling
error. The ABC News/Washington
Post survey shows Giuliani doing consistently better, and Thompson
doing consistently worse, than the automated surveys conducted by Rasmussen
Reports, although the discrepancy has been largest in terms of how the most recent ABC/Post poll compares to Rasmussen surveys conducted over the last month or so.
To try to answer Andrew's question, it makes sense to take
two issues separately. First, why are
the surveys producing different results for the Republican primary?
At the most basic level, these surveys seem to be measuring
the same thing: Where does the Republican nomination contest stand nationally? And
both surveys begin with a national sample of working telephone numbers drawn
using a random digit dial (RDD) methodology. Take a closer look, however, and
you will see some pretty significant difference in methodology:
ABC/Post survey uses live
interviewers. Rasmussen uses an automated recorded voice that asks
respondents to enter their answers by pushing buttons on a touch tone
keypad. This method is known as Interactive Voice Response (IVR). The
response rates -- and more importantly, the kinds of people that respond --
are likely different, although neither pollster has released specific
response rates for any of the results plotted above.
ABC/Post survey attempts to
select a random member of each household to be interviewed by asking "to
speak to the household member age 18 or over at home who's had the last
birthday" (more details here). Rasmussen interviews whatever adult member
of the household answers the telephone. Both organizations weight the
final data to reflect the demographics of the population.
Reports weights each survey by
party identification, using a rolling
average of recent survey results as a target (although their party
weighting should have little effect on a sub-group of Republican primary
voters). The ABC/Post survey
does not weight national surveys at this stage in the campaign by party
[Update -- one I overlooked: The ABC/Post survey includes Newt Gingrich on their list of choices. Gingrich receives 7% on their most recent survey. If the Rasmussen survey prompts Gingrich as a choice, they do not report it. It is also possible that Rasmussen omits other candidates as well, as t Their report provides results for just Giuliani, Thompson, Romney and McCain. Update II -- Scott Rasmussen informs via email: "We include all announced candidates plus Fred Thompson"].
perhaps most important for Andrew's question: The ABC/Post survey asks the presidential primary question of all
adults that identify with or "lean" to the Republicans. The Rasmussen
survey screens to a narrower slice of the population: Those they select as
"likely Republican primary voters."
Unfortunately, neither pollster tells us the percentage of
adults that answered their Republican primary question, but we can take a
reasonably educated guess: "Leaned Republicans" have been somewhere between 35%
and 42% of the adult population on surveys conducted in recent months by Gallup and the Pew
Research Center. If Rasmussen's likely voter selection model for Republican
is analogous to their model
for Democrats, their "likely Republican primary" subgroup probably
represents 20% to 25% of all adults.
Consider also that, even before screening for "likely
voters" and regardless of the response rate,
those willing to complete an IVR study may well represent a population that is
better informed or more politically interested than those who complete a survey
with an interviewer.
Put this all together, and it is clear that the Rasmussen
survey is reaching a very different population, something I would wager
explains much of the difference in the results charted above.
Now, the second question, which result is more "accurate?" It is tempting to say that this question is impossible to
answer, since we will never have a national primary election to check it against.
But a better answer may be that "accuracy" in this case depends on what we want
to use the data for.
If we were trying to predict the outcome of a national
primary, and if all other aspects of methodology were equal (which they're
not), I would want to look at the narrower slice of "likely voters" rather than
all adult "leaned Republicans." Since the nomination process involves series of
primaries and caucuses starting with Iowa and New Hampshire, and since
the results from those early contests typically influence preferences in the
states that vote later, we really need to focus on early states for a more
"accurate" assessment of where things stand now. While interesting and fun to
follow, these national measurements provide only indirect indicators of the
current status of the race for the White House.
Why would the ABC/Post survey want to look at all
Republicans, rather than likely voters? Here is the way ABC polling director
Gary Langer explained it in his online
column this week:
I like to think there are two things we cover in
an election campaign. One is the election; the other is the campaign.
The campaign is about who wins. It's about tactics
and strategy, fundraising and ad buys, endorsements and get-out-the-vote
drives. It's about the score of the game - the horse race, contest-by-contest,
and nothing else. We cover it, as we should.
The election is the bigger picture: It's about
Americans coming together in their quadrennial exercise of democracy - sizing
up where we're at as a country, where we want to be and what kind of person
we'd like to lead us there. It's a different story than the horse race, with
more texture to it, and plenty of meaning. We cover it, too.
We ask the horse race question in our national
polls for context - not to predict the winner of a made-up national primary,
but to see how views on issues, candidate attributes and the public's personal
characteristics inform their preferences.
Questions like Andrew's are more consequential in the statewide surveys we
are tracking here at Pollster.com, and those surveys have been producing some
discrepancies even bigger than the one charted above. We will all be in a
better to make sense of those differences if we know more about the
methodologies pollsters use. I'll be turning to that issue in far more detail
A new American Research Group statewide survey of likely voters in Arizona (conducted 7/23 through 7/26) finds:
Among 400 Republicans, Sen. John McCain leads former Mayor Rudy Giuliani (32% to 23%) in a statewide primary; former Sen. Fred Thompson trails at 15%, former Speaker Newt Gingrich and former Gov. Mitt Romney both at 7%. All other candidates receive less than 5% each.
Among 400 Democrats, Sen. Hillary Clinton leads Sen. Barack Obama (39% to 25%) in a statewide primary; Gov. Bill Richardson trails at 9%, former Sen. John Edwards at 8%. All other candidates receive 5% or less.
A new SurveyUSA automated survey of 514 registered voters
in Missouri (conducted 7/24 through 7/25) finds
State Attorney General Jay Nixon leading Gov. Matt Blunt
(57% to 38%) in a statewide gubernatorial general election
A new KCCI/KCRG statewide survey (via MyDD) of likely caucus voters in Iowa (conducted 7/23 through 7/25 by Research 2000) finds:
Among 400 Democrats, former Sen. John Edwards edges out Sen. Hillary Clinton (27% to 22%) in a statewide caucus; Sen. Barack Obama trails at 16%, Gov. Bill Richardson at 11%. All other candidates receive less that 5% each.
Among 400 Republicans, former Gov. Mitt Romney leads former Sen. Fred Thompson (25% to 14%) in a statewide caucus; former Mayor Rudy Giuliani trails at 13%, Sen. John McCain at 10%, former Speaker Newt Gingrich at 6%. All other candidates receive less than 5% each.
The survey also includes general election match-ups, job approvals, and questions regarding Iraq policy.
Earlier today on Hardball, MSNBC's Chris Matthews had the
following exchange with Howard Wolfson, communications director for the Clinton campaign (my
transcript below the video):
Matthews: How would you describe [Sen. Clinton's] position
in voting to authorize the war in Iraq believing we weren't going to
war, that Bush really didn't intend to go to war. Was that naive?
Wolfson: Look, she's taken responsibility for the
vote. She's been asked about this...
Mathews: Wouldn't you call that naive to believe...
Matthews: ...that we're not going to
war when everybody thought we were going to war? I
thought we were going to war.
Wolfson: I guess 80 percent of the country was naive
Matthews: They didn't think Bush
would take us to war?
Wolfson: I think people were, believed George Bush was
going to do what he said he was going to do, which was to try diplomacy. And he didn't.
Matthews: Anybody who didn't think
we were going to war, in the months leading up to the war in Iraq, wasn't paying attention.
Did 80% of Americans believe that President Bush would "try
diplomacy" in an effort to avoid a war with Iraq?
I checked the Iraq
archives at the Polling Report. Most
of the questions asked in late 2002 focused on whether and under what
conditions Americans would support going to war. However, a CNN/USA Today/Gallup survey of 1,017 adults conducted November 22-24,
2002 (a month after the vote to authorize the use of military force in Iraq), showed
that a majority of Americans believed President Bush had "already decided" to
Which comes closer to your view
about President Bush? [Rotate:] Bush has already decided to invade Iraq and has agreed
to UN inspections mainly to gain international support for that action. OR,
Bush has not yet decided whether to invade Iraq
and has agreed to UN inspections mainly to determine if an invasion of Iraq were
58% - Bush has already decided to
38% - Bush has not yet decided
whether to invade
4% - No opinion
A bit of context on the timing: The U.S.
Senate passed the resolution
authorizing the use of military force against
on October 11, 2002 (with Senator Clinton voting in favor). President Bush signed it into law on October
16. Three weeks later, on November 8,
the United Nations Security Council unanimously passed Resolution
1441 urging Iraq
to "comply with its disarmament
obligations" or face "serious consequences." Iraq
to the resolution on November 13, and under its terms, U.N. weapons inspectors
were set to return to Iraq
on November 27 after a four year absence to conduct onsite inspections in
search of weapons of mass destruction.
Gallup fielded its survey on November 22-24, just days
before the return of the U.N. inspectors, a time when one might expect optimism
regarding the use of diplomacy to resolve the conflict. Yet even then, 58% of Americans believed the
President had "already decided to invade Iraq."
*With apologizes to Frank Newport, who has made "unspinning"
questionable assertions about public opinion on Sunday morning talk shows a
regular staple of his Gallup Guru
A few weeks ago, a reporter called to ask if I thought New York City
Mayor Michael Bloomberg might be conducting polls to assess a potential
independent candidacy for President. Needless to say, I have no way of knowing.
Bloomberg is under no legal requirement to report on his personal
expenditures, and as one of the wealthiest men in American, can certainly
afford to spend his own money to commission polls.
I mentioned to the reporter that the only way we might
discover that Bloomberg is polling is on the off chance that a respondent might
report having participated in a Bloomberg-centric survey and post a comment on
a blog somewhere (something that seems to happening more and more lately). As such, this comment from Open Left's
Chris Bowers is very intriguing:
Mountain West Research
is a survey call center and, in this most recent case, was likely the vendor used by a political
pollster to conduct interviewing. The link above involves a poll conducted in
June 2006, when Lieberman's pollster of record was the Democratic firm Greenberg,
Quinlan, Rosner Research. As I assume that Mountain West conducts surveys for many
different political pollsters, the Lieberman association is probably not a good
indicator of who sponsored this particular survey.
Update I - Stan
Greenberg responds via email:
We have absolutely nothing to do
with a recent survey for Bloomberg, though we use Mountain West and have
surveys in the field there regularly - along with Sun Surveys, our own company.
Update II - There
may be less to this story than meets the eye. Also via email, Chris Bowers adds
They only asked two questions about
Bloomberg, but they came after a long battery opinion of the two parties. The
rest of the questions, or about 30-40% of the call, were about education and No
Child Left Behind. It is not clear to me that it definitely was a Bloomberg
Given that description, the survey could have been conducted
by any number of entities not affiliated with Bloomberg, including an education
interest group or any group wondering about the potential impact of an independent
presidential candidacy. It is also possible that this was an "omnibus"
survey that included questions on different subjects paid for by several different clients.
Consider also that if Bloomberg had paid a campaign pollster to
evaluate his potential as an independent candidate, the survey would have had
many more questions about Bloomberg, including "message tests" involving his biography
and positions on issues.
**Presumably, "Chris" is Open Left blogger Chris Bowers.
A new Diagio/Hotline national survey (story, results) of 801 registered voters (conducted 7/19 through 7/22) finds:
33% approve of the job George Bush is doing as president; 63% disapprove.
35% think that the next president will start withdrawing troops from Iraq "within three months, with all troops out within nine months;" 29% think that he or she will "leave a substantial number of troops in Iraq, but have them concentrate on training Iraqis and targeting Al Qaeda leaders in Iraq;" 14% say he or she will "begin an immediate withdrawal of all U.S. troops in Iraq;" 7% say he or she will "make no cutbacks in U.S. troops in Iraq."
Among 122 likely Republican primary voters, former Mayor Rudy Giuliani edges out former Sen. Fred Thompson 20% to 19%; Sen. John McCain runs at 17%, former Gov. Mitt Romney at 8%. All other candidates receive less than 5% each.
Among 393 likely Democratic primary voters, Sen. Hillary Clinton leads Sen. Barack Obama 39% to 30%; former Sen. John Edwards trails at 11%. All other candidates receive less than 5% each.
A new Pew Research Center national survey (story, results) of 1,040 adults (conducted 7/20 through 7/23 by Opinion Research Corporation) finds:
In an open ended question, 42% say Sen. Hillary Clinton is the 2008 presidential candidate they've "heard the MOST about in the news lately;" 22% say Sen. Barack Obama, 3% say Pres. George W. Bush. 2% each say former Sen. John Edwards, Sen John McCain, former Mayor Rudy Giuliani, former Sen. Fred Thompson, and former Gov. Mitt Romney.
"Four-in-ten Republicans say news organizations are giving too much coverage to the campaign. Only 19% of Democrats feel the same way."
New Gallup anaylses (and video) of recent national surveys of 3,006 adults (conducted in June and July) find:
Among 1,204 Republicans and those who lean Republican, former Mayor Rudy Giuliani leads former Sen. Fred Thompson (29% to 19%) in a national primary. Among the 480 who attend church every week, Giuliani edges out Thompson 24% to 20%. Among the 420 who seldom or never attend church, Giuliani leads 33% to 18%.
Among 1,515 Democrats and those who lean Democratic, Sen. Hillary Clinton leads Sen. Barack Obama (35% to 22%) in a national primary. Among the 364 who attend church every week, Clinton leads Obama 39% to 24%. Among the 794 who seldom or never attend church, Clinton leads Obama 32% to 20%.
In general election match-ups, Giuliani leads Clinton by more than ten points among Americans who attend church weekly and among those who attend nearly weekly or monthly. Among Americans who seldom or never attend church, Clinton leads Giuliani by more than ten points (54% to 43%).
A new Alabama Education Association statewide survey of likely voters in Alabama (conducted 7/11 through 7/13 and 7/16 through 7/19 by Capital Survey Research Center) finds:
Among 396 Republicans, former Sen. Fred Thompson leads former Mayor Rudy Giuliani (34% to 20%) in a statewide primary; Sen. John McCain trails at 11%, former Speaker Newt Gingrich at 7%, former Gov. Mitt Romney trails at 5%. All other candidates receive less than 5% each.
Among 371 Democrats, Sen. Hillary Clinton edges out Sen. Barack Obama (33% to 29%) in a statewide primary; former Sen. John Edwards trails at 9%, former V.P. trails at 6%. All other candidates receive less than 5% each.
This morning, The
Washington Post's Peter Baker takes a closer
look at why President Bush, with a disapproval rating of 65% on the latest
Post/ABC survey, "is in the running for most unpopular president in the history
of modern polling." While poll junkies ought to read the whole thing, this
bit of historical context from pollster Patrick Caddell is especially
"It's astonishing," said
Pat Caddell, who was President Jimmy Carter's pollster. "It's hard to look
at the situation today and say the country is absolutely 15 miles down in the
hole. The economy's not that bad -- for some people it is, but not overall. Iraq is terribly handled, but it's not Vietnam; we're
not losing 250 people a week. . . . We don't have that immediate crisis, yet
the anxiety about the future is palpable. And the feeling about him is he's
irrelevant to that. I think they've basically given up on him."
Baker goes on to float the theory that "the changing nature
of society," and in particular of the way Americans receive their news, is
partly responsible for the difference:
"A lot of the commentary that
comes out of the Internet world is very harsh," said Frank J. Donatelli,
White House political director for Ronald Reagan. "That has a tendency to
reinforce people's opinions and harden people's opinions."
Additional results from the recent ABC News/Washington Post national survey (ABC story, results; Poststory, results) of 1,125 adults (conducted 7/18 through 7/21) finds:
Among Republicans and those who lean Republican, former Mayor Rudy Giuliani leads Sen. John McCain (34% to 16%) in a national primary; former Sen. Fred Thompson trails at 14%, former Gov. Mitt Romney at 8%, former Speaker Newt Gingrich at 7%. All other candidates received less than 5% each.
13% of Republicans are very satisfied with the choice of candidates for the Republican nomination; 53% are somewhat satisfied, 26% are somewhat dissatisfied, and 6% are very dissatisfied.
New polls have moved noticeably above President Bush's recent low point of approval, though the current trend estimate remains below 30%. New polls have generally fallen at 30% approval and above, though one is as low as 25%. This change of trend was first discussed about a week ago here. Since then, the evidence for a shift in trend has mounted.
The ARG poll, taken 7/18-21/07, sets the low point at 25% approval, 71% disapproval. Gallup, done 7/12-15/07 got approval at 31%, disapproval at 63%. CBS/New York Times has two recent results: 7/9-17/07 has approval at 29%, disapproval at 64%, while the newest CBS/NYT poll of 7/20-22/07 puts approval at 30%, disapproval at 62%. Fox, 7/17-18/07 has approval at 32%, disapproval at 61%, while ABC/Washington Post 7/18-21/07 estimates approval at 33% and disapproval at 65%. So that puts four of the last six polls above 30% with only one in the mid-20s, where polling in late June was falling.
The trend estimate now stands at 29.6%, and the slope of the trend has clearly begun to bend from a steep downward trend to a less negative one. This is classic behavior of my standard "old blue" estimator. It takes a while to change direction, and while it does it slowly bends until it shifts direction entirely. In contrast, the "ready red" estimator is more sensitive and picks up on direction changes more quickly. The red line in the figure above shows a sharp change at just under 29% to a current estimate of just under 31% approval. The red line is often too sensitive, mistaking short term random noise for real change. However, we've now accumulated enough supporting polls to make me more confident that the upturn in the red estimator is probably real. At the same time, this does not mean the trend will remain as sharply up as the red estimator seems to suggest. That won't be known for a while yet, as more polls help define what the path of the next month or so is.
Recent polls have fallen above and below the trend estimate, and at the moment none of the last 10 polls constitutes an outlier, though there are both high and low polls rather close to the confidence interval limits. This range of results is also reflected in the very wide gray area at the end of the trend in the Bootstrap plot below. The wide gray area reflects our uncertainty about where approval is right now in a time of change in direction.
Finally, the last plot below shows the most recent 20 estimates of approval. The red dots trace out the clear decline, low point, and move back upwards in the "old blue" estimator. So even that estimator is feeling a change.
A few more thoughts about the various efforts to determine
who "won" last night's CNN/YouTube debate. In addition to the SurveyUSA panel-back
survey I blogged
last night, there were (by my count) at least four different focus groups
convened in various locations:
reported last night on two "dial groups," one in Manchester,
New Hampshire and another in Las
Vegas, Nevada, in which selected voters constantly adjusted a dial that
rates their reaction to whatever they are watching. (coverage here, here
group tester Rich Thau (Messagejury.com)
did another dial group among 12 likely voters in Bedford New Hampshire. [Update: more on the Thau groups here, via First Read].
News had Republican pollster Frank Luntz conduct what appeared to be a
more traditional focus group (albeit with a larger than traditional group
conducted on live television) in Charleston,
South Carolina (transcript,
video available from this page). [Update: A reader emails to say that the Luntz group also involved a "dial test"].
Because they are more qualitative, the "results" of these
efforts can sometimes be in the eye of the beholder. Since they involve
small, non-random samples of different kinds of voters, they frequently produce
contradictory results. Some researchers like to use bigger groups (of
40-50 participants), but that approach increases the risk of creating a "group
dynamic" that allows the most opinionated participants to sway others toward
their point of view. We also know little from the various reports linked to
above about the kinds of voters recruited for each group. How many were completely
undecided before the groups? How many were political independents? The answers
to those questions might tell us a lot about the reported outcome in each case.
Not surprisingly, last night's focus groups identified many different "winners." According to coverage from NBC's First Read, "Barack Obama
got the most favorable [response] in terms of the best performance" in the CNN New
Hampshire group, but their Nevada
group showed "Bill Richardson and Hillary Clinton in a tie with six votes
each." On the basis of favorable reaction from his focus group, Frank Luntz
concluded that "Obama will be shown as the winner of tonight's debate."
Meanwhile, depending on which source we turn to, the Rich
Thau group of 12 New Hampshire voters produced good news for John Edwards, Barack Obama and Joe
Biden. Under the headline, "Very Much an Edwards Night," Thau reports
that "a plurality of our group (four of 12) thought Sen. Edwards won the
debate." However, NBC's First Read tells us,
"his survey had Obama doing the best (in terms of improvement from pre-debate
to post-debate; Obama and Edwards tied with the highest post-debate score)." And
Thau's before-and-after "comfort scores" shows
the biggest gains by Joe Biden.
If you're having trouble making sense of all this, you're
Also, I am guessing that the audience for the YouTube debate,
while probably larger than usual for CNN, was still a tiny sliver of those who will ultimately vote in Democratic primaries or caucuses. Assuming the ratings followed typical patterns, we
should assume that far more voters saw or read news accounts about the debate
this morning than experienced it in full last night.
In that regard, consider this very sensible observation
made last night by Chris Bowers, on the liberal site Open Left:
I noticed that both on CNN, and in
the progressive blogosphere, several people started trying to determine who
won, and even making declarations that Candidate X had, in fact, clearly won
the debate. I was taken aback. There was a winner? There was even a
competition? What did the winner, well, "win?" New hard-core voters
in the Iowa and New Hampshire primaries? New donors? The
deliverance of a biting comment that will psychological scar one of his or her
opponents so badly that s/he cannot continue on the campaign trail with naught
but a thoroughly shredded sense of self-esteem?
The truth is, there is only one objective way for any candidate to
"win" the debate: if the debate, or the post-debate spin,
cumulatively result in your campaign moving closer to taking the nomination
than you were before the debate. That's it. From this "God's eye"
perspective, it is doubtful that anyone won the debate. The factors that make
the most difference in pushing someone closer to the nomination are, first, new
Iowa supporters and, second, new supporters in
Everything else is pretty distant, and I seriously, seriously doubt anyone
picked up much, if anything, in those two categories
Among 457 likely Democratic primary voters in Florida, Sen. Hillary Clinton (at 46%) leads Sen. Barack Obama (15%) and former Sen. John Edwards (13%) in a statewide primary (conducted 7/18 through 7/19).
Among 490 likely Republican primary voters in Florida, former Mayor Rudy Giuliani edges out former Sen. Fred Thompson (22% to 21%) in a statewide primary; Sen. John McCain and former Gov. Mitt Romney both trail at 13% (conducted 7/18 through 7/19).
Among 1,461 likely voters nationwide, Clinton leads Rep. Ron Paul (49% to 34%) in a general election match-up; Obama leads Paul 50% to 30% (conducted 7/20 through 7/22).
View all Florida Primary poll data at Pollster.com:
partly in response to a critique
offered here in May, SurveyUSA fielded an automated "panel-back"
survey among 717 Americans who said they watched tonight's CNN/YouTube
debate. The idea of a panel-back survey is to interview the same sample before
and after some event, allowing for an individual level measurement of change. With
a panel back survey, a pollster can determine how many respondents shifted
Here is the
key question asked of these respondents before and after the debate:
Regardless of who you may vote for,
and regardless of whether or not you would vote for a Democratic candidate for
President, which Democratic candidate would make the best President of the United States?
Americans who watched the YouTube Democratic Debate 7/23/07
on CNN went into the debate thinking Hillary Clinton would make the best
President of the 8 Democrats on stage, and came out of the debate even more
convinced, according to an exclusive SurveyUSA poll. Before the debate, 40% saw
Clinton as the
most Presidential Democrat; after the debate, when the same respondents were
re-interviewed, 43% saw her as the most Presidential Democrat
It is not
obvious from the before
and after snapshot, but it appears that most of those who responded to both
calls were consistent in their answer to this question, before and after. To
the extent that there was a shift, however, it worked slightly to the benefit
of Clinton and Biden, slightly against Barack Obama.
asked if respondents "viewed each Democrat positively or negatively" before and
after the debate. The results indicate that exposure to the debate improved
perceptions of all the candidates except for Gravel. The summary reported by
SurveyUSA (with bullets added):
went up 38 points, from Minus 6 to Plus 32.
went up 17 points, from Plus 24 to Plus 41.
Clinton went up 16 points, from Plus 34 to
went up 16, from Plus 22 to Plus 38.
went up 15, from Minus 21 to Minus 6.
Richardson went up 14, from Minus 1 to Plus
went up 7, from Minus 21 to Minus 14.
went down 3, from Minus 29 to Minus 32
does include some important caveats worth noting. The most important is that
this sample of 717 is not a random sample of all Americans (or likely voters),
or even a random sample of all Americans who happened to be at home this
evening. They started with a first round of calls to a random sampling of
households, but this final pool of 717 reflects only those who responded to the
first survey, reported that they would watch the debate and take a second call and
then actually completed a second call. The summary also notes that "panel
back is not universally sanctioned" as a tool to measure debate reaction,
pointing to the unsparing criticism by pollster David Hill. Writing for The
Hill in 2004, he described the panel-back as "worthless:"
Considerable scholarly research
demonstrates that simply being interviewed renders an otherwise normal voter
abnormal. After being polled, voters are much more likely to seek out political
information through the media, discuss politics with others and eventually to
vote. The known effects are so great that in the earliest days of polling,
voters would be screened at the outset of an interview to ascertain if they had
ever been interviewed before.
I don't share Hill's blanket aversion to panel-back surveys,
but he is right to warn that the first interview can add some artificiality to
either the nature of the sample or the way those surveyed watch and react to the
debate. For all its drawbacks, however, this approach is a great improvement to
simply calling a fresh sample of those at home and asking "who won" without any
data on which candidate they supported beforehand.
In the spirit of better understanding the data, I do have a
few questions for the analysts at SurveyUSA:
First, what percentage of the 717 debate watchers provided
the same answer to the first question before and after the debate?
Second, the summary reports:
SurveyUSA dialed at random into all 50 states. Respondents
were asked if they planned on Monday night, 7/23/07, to watch the entire
YouTube debate on CNN. Those respondents who said "yes" were then
asked if SurveyUSA could call them back, immediately after the debate
What percentage of those adults interviewed on the first
call agreed to watch the debate
and be interviewed a second time?
did SurveyUSA have any way to confirm that they interviewed the same individuals
within each household before and after the debate?
The blogs and mainstream media have featured much speculation,
as they always do, on the trends in trial-heat vote questions on national and
early state surveys and what they may foretell about the likely outcome next Spring.
I thought a review of what these same surveys were saying four years ago about
the Democratic nomination contest might serve as a valuable reality check.
According to the Polling Report (my source
for all the data that follows), seven organizations released polls in July 2003
on the Democratic race. The "undecided" option generally led on most surveys, and
future nominee Sen. John Kerry received just 14% of the vote, typically slightly
behind or tied with Rep. Dick Gephardt or Sen. Joe Lieberman. Howard Dean, who
would emerge as a perceived "frontrunner" over the summer months was still
averaging just 11%. Moreover, as the chart of the Annenberg National Election Survey
data (described in more detail here several months ago) demonstrates vividly, Dean
remained mired in the teens nationally until December 2003, but then plunged
dramatically after his third place finish in Iowa.
Was it any different in the early states this time four
years ago? Dean, whose campaign had started running television advertising in Iowa and New
Hampshire in June, had increased his share of the
vote as compared to polls conducted in both states earlier in the year. Two
polls - one from the Des Moines Register and another internal poll later
released by Dick Gephardt's campaign - both showed Dean and Gephardt leading with
just over 20%. Meanwhile, John Kerry and John Edwards, the two candidates that
would ultimately emerge to finish first and second in the caucuses with a
combined 69% of the state delegates selected, receiving an average of 18% of
the vote in the surveys take in July 2003.
The New Hampshire
polls of July 2003 probably came closest to the ultimate result in that state,
although John Kerry's average in those polls (24%) fell far short of his
ultimate share of the vote (38%). Moreover, John Edwards and Wesley Clark who
each averaged just 2% in these early polls, ultimately received just over 12%
of the vote.
Of course, 2008 is stacking up as a very different year than
2004. Hillary Clinton and Barack Obama are arguably better known now than
any of the 2004 candidates were at this point four years ago. My point here is not to argue that 2008 will be like 2004, however, but to try to remind readers that there is still considerable room for change in the preferences voters express on these early surveys. We are just now coming to the close of the first phase of the campaign
in the early states, where (except for Bill Richardson and Mitt Romney) none of the candidates have yet run significant amounts of television
advertising. While the contest has started a bit earlier, we still have a long
way to go.
Additional results from the recent ABC News/Washington Post national survey (story, results) of 1,125 adults (conducted 7/18 through 7/21) finds:
33% approve of the job George Bush is doing as president; 65% disapprove -- "a number surpassed only by Richard Nixon in the summer of 1974, albeit matched by Harry Truman and approached by Bush's father."
37% approve of the job Congress is doing; 60% disapprove.
55% trust Democrats in Congrss "to do a better job handling the situation in Iraq;" 32% trust Bush.
A new Quinnipiac University statewide survey of 1,106 registered voters in Florida (conducted 7/12 through 7/16) finds:
Among 438 Democrats, Sen. Hillary Clinton (at 36%) leads Sen. Barack Obama and former V.P. Al Gore (both at 14%) in a statewide primary; former Sen. John Edwards trails at 9%. All other candidates recieve less than five percent.
Among 433 Republicans, former Mayor Rudy Giuliani (at 30%) leads former Sen. Fred Thompson (18%) in a statewide primary; Sen. John McCain trails at 10%, former Gov. Mitt Romney at 9%, former Speaker Newt Gingrich at 5%. All other candidates receive less than five percent.
General election match-ups:
Giuliani 46%, Clinton 44%
Giuliani 39%, Clinton 41%, Bloomberg 9%
Giuliani 47%, Obama 39%
McCain 40%, Clinton 46% McCain 38%, Obama 42%
View all Florida Primary poll data at Pollster.com:
A new ABC News/Washington Post national survey (ABC story, results; Poststory, results) of 1,125 adults (conducted 7/18 through 7/21) finds:
Among Democratic voters and those who lean Democratic, Sen. Hillary Clinton leads Sen. Barack Obama (39% to 28%) in a national primary; former Sen. Al Gore trails at 14%, former Sen. John Edwards at 9%. Without Gore, Clinton leads Obama 45% to 30%. All other candidates receive less than five percent.
Among those who support Clinton, 68% "strongly" support her while 32% "somewhat" support her. Among those who support Obama, 56% strongly support him while 43% somewhat support him.
83% are satisfied "with the choice of candidates for the Democratic nomination for president this year;" 16% are dissatisfied.
The biannual health study, which
provides data on more than 42,000 of the state's roughly 12 million households,
is used to "drive decision-making and to drive the recommendations we make
to the Legislature or to the governor," said Sandra Shewry, director of
the California Department of Health Care Services.
If data are inaccurate, a survey paints a nonrepresentative picture of
populations, which can in turn be used as the basis for an improper allocation
of funds or just bad decisions, she said.
"The implications are that if we do not include people with cellphones
only, then we are likely to be underreporting un-insurance rates, and we're
likely to underreport smoking prevalence rates in the state," survey
director David Grant said. The numbers won't be dramatic, he said, "but it
will introduce some level of bias in our estimates.