September 23, 2007 - September 29, 2007
Langer remembers the numbers for Newt Gingrich when he was Speaker.
Newport looks at Gallup's
numbers on the Democratic presidential race with and without Al Gore.
Agiesta looks at what the Fox
News poll has to say about how closely Americans followed the Moveon.org "General
Cost considers how voter inattention combines with horserace coverage to drive
early trial-heat poll numbers.
Ambinder and Jonathan
Martin report the latest poll memo from the Romney campaign.
Hill looks back at what polls in 1994 had to say about Hillary Clinton's
health care initiative.
Mellman looks at how much results can change when pollsters weight the data.
Smith has video of a verbal smackdown between Clinton pollster Mark Penn and Romney media
consultant Alex Castellano.
reader "sjc" notices something I overlooked: How many more respondents are
opting for the Democratic primary in New
Hampshire in the latest CNN/WMUR/UNH poll now as
compared to 2000.
Since kicking off our Disclosure
Project on Monday, we are pleased to report some very favorable early
mentions and links from a variety of bloggers including The Atlantic's Marc
Ambinder, Time's Ana
Marie Cox, USA Today's Memmott
and Lawrence, Politico Ben
Smith, MSNBC's Clicked,
MyDD's Jerome Armstrong and Jonathan Singer, The Democratic Strategist's Ed
Kilgore. Other names you may not recognize have left comments
or endorsed the efforts on their blogs.
As of yesterday, we can add an important name to that list: My
colleague Nancy Mathiowetz, the
current president of the American Association for Public Opinion Research
(AAPOR), left the following comment here at Pollster.com:
As President of the American Association for Public
Opinion Research, I believe that this is an excellent opportunity for public
opinion researchers to help improve the public understanding of polling
methodology and interpretation.
Pollster.com is to be applauded for this effort.
More information from AAPOR about disclosure:
Regular readers will know that I serve with Mathiowetz on AAPOR's Executive
Council, but it is nonetheless an honor to have her support.
I want to recognize the prompt and complete replies we have already received
to our queries from the pollsters at ABC News/Washington Post and LA Times/Bloomberg
and Time. Other organizations have
requested more time to gather and report the data we requested. Given that we
are doing something new here while also "debugging" our own process, we are
going to allow as many organizations as possible to respond before publishing
the first set of replies (and before sending out similar queries for polls done
in New Hampshire, South Carolina and the nation as a whole).
I have also been in contact with the pollsters at Quinnipiac, Mason-Dixon
and InsiderAdvantage regarding my post last Friday on their recent polls in Florida and will have
more on that subject very soon.
I can say that all of those that have responded appear to be making a good
faith effort to be transparent and, as Nancy Mathiowetz put it, "help improve
public understanding of polling methodology and interpretation." For that we
However, not all of the pollsters have responded to our queries, which is
your why your support is important. As Ana Marie Cox put it:
The questions will probably seem unbearably wonky
to many, but the reason for wanting the answers is important: They'll help
reporters (and readers) better judge the accuracy and importance of these
primary state polls.
Or to quote DemfromCT from DailyKos:
The bottom line is that if you want better data to
analyze, then we, the consumers of all things political, ought to support
pollster.com in asking for it. And if we expect and appreciate the analysis
done by pollster.com, Swing State Project, Open Left, Slate, Real Clear
Politics or any of the other sites that digest and analyze polling data, let's
help make the data a bit more "open source" and transparent.
If you can comment or blog your endorsement of this, we would greatly
A new Fox News/Opinion Dynamics national survey (story, results; OJ story, results) of 900 registered voters (conducted 9/25 through 9/26) finds:
- 34% approve of the job George Bush is doing as president; 58% disapprove.
- 45% of Republicans are "hoping there is someone new who you haven't heard about yet who may enter the 2008 presidential race;" 49% are not. 36% of Democrats are hoping for it; 60% are not.
- General election match-ups
Clinton 46%, Giuliani 39%
Clinton 42%, Giuliani 32%, Bloomberg 7%
Clinton 48%, Thompson 35%
Clinton 46%, McCain 39%
Obama 41%, Giuliani 40%
Obama 45%, Thompson 33%
Obama 40%, McCain 38%
A new Research 2000 statewide survey of 600 likely voters in Texas (conducted 9/24 through 9/26 for Daily Kos) finds:
- If the 2008 election for U.S> Senate were held today, 40% would re-elect Republican Sen. John Cornyn, 15% would consider voting for another candidate, and 35% would vote to replace him.
- Cornyn leads Democratic state Rep. Rick Noriega (51% to 35%) in a hypothetical general election match-up.
If polls are political crack, than I've been happily
snorting the latest from the CNN/WMUR New Hampshire primary polls conducted the
University of New Hampshire (also known as the
"Granite Poll") for the last 24 hours or so. I've become a fan of this survey because
of the "internals" included in their questionnaire and the very helpful set of crosstabs
(for the Democratic
samples) released as always by University
of New Hampshire Survey Center.
on the two surveys highlighted the same trial heat numbers that virtually
everyone else is focusing on: Clinton's
lead has increased since July, while support for Mitt Romney has fallen off so
that he now runs neck-and-neck with Rudy Giuliani.
However, to elaborate on a point I tried to make a week
ago, the trial heat numbers in such polls (yes, the ones we track here at
Pollster) may be the least useful at this stage, particularly if you interpret
them as a prediction of what voters will do several months from now. Surveys
are good at telling us what voters think and who they currently prefer, but those
preferences are always subject to change. The latest survey of Democrats from
UNH/CNN/WMUR helps put some of those preferences in perspective. Here a few
1) A preference is still not a final decision -- Just
before asking the likely Democratic primary voter who they would vote for "if
the election were held today," the UNH pollsters ask them how close they are to
making a final decision.
So even though 91% can name a candidate they would support,
only 17% are "definitely decided," 28% are just "leaning to someone" and more
than half (55%) "still trying to decide." Of course, voters may ultimately
decide to support their current preference, but by their own report most have
not yet made that final choice.
I wrote the last two paragraphs before the Republican results
had been released, but those numbers show even more evidence of uncertainty. Although
84% of Republicans express a candidate preference, 66% say they are "still
trying to decide" while only 13% are definitely decided and 21% are leaning to a
2) A Big Shift on
"Can Win"- One of the biggest and most noteworthy shifts on the Democratic
poll involves the question about which candidate "has the best chance of
beating the Republican nominee in the general election next November?" Although.
Hillary Clinton's share of the vote has increased four points (from 39% to 43%)
since June, we see a 17-point increase (from 37% to 54%) in assessments that
she has best chance to win in November:
These results are consistent with similar findings from the
most recent NBC/Wall Street Journal national survey
of Democrats, which show the percentage choosing Clinton as the Democrat with
the "best chance to defeat the Republican candidate" growing from 39% to 54%
from April to September. Also, the Pew
Research Center shows 53% of Democrats now naming Clinton as the candidate
with the "the best chance of winning the general election against a Republican"
(although they had not asked the question previously).
I noticed something interesting buried in the cross-tabs. Most of the
shift on this measure in the UNH polls has occurred among the roughly 60% of
likely Democratic primary voters that are less than "extremely interested" in
the primary. That suggests a common pattern: Less attentive voters reacting mostly
to the dominant campaign news story, "horserace" coverage relentlessly portraying Clinton as front runner consolidating her lead.
One thing to keep in mind though. Electability ratings are
not always a great predictor of, well...of electability. The 2004 National Annenberg
Election Study (NAES)
tracked electability ratings of each of the Democratic candidates (how likely respondents
considered each candidate to beat George Bush in November) among likely
Democratic primary voters from October 2003 though February 2004. In a paper presented
to the American Political Science Association annual meeting in September 2004),
Annenberg's Kate Kenski
Dean's electability and viability
ratings were higher than those of Kerry until the Iowa caucuses. After the Iowa caucuses, Kerry's electability ratings
surpassed those of Dean. Impressions of Dean's electability against Bush
reached a high point
of almost 53% around November 19. Two months later, on the evening of the Iowa caucuses, this
perception had decreased eight points.
3) A McCain
Resurgence? Support for John McCain among likely Republican primary voters,
which had fallen sharply in New
Hampshire as elsewhere, has increased to 17% on this survey
from a low of 12% in July. McCain's support had hovered around 30% in UNH polls
conducted during 2006 and early 2007.
An outlier? Perhaps, but dig deeper and the survey yields
evidence of some underlying strengths for McCain in New Hampshire that may provide a foundation
for a future resurgence there. His favorable rating now (63% favorable, 24%
unfavorable) is slightly but not significantly better than it was in February (59%
favorable, 27% unfavorable) when he had 28% of the vote and ran a point ahead
of Giuliani. And McCain now leads the Republican field (with 32%) on the
question of which candidate has the "right experience to be president."
It bears repeating: Eight years ago, John McCain defeated
George Bush by a huge margin in New
Hampshire (49% to 30%) among these same voters. Voters
have a way of falling back to past preferences.
4) Favorable Ratings
- Talk to campaign pollsters about the value of the trial heat results and most
will tell you a similar story: Vote preference is usually the last thing to change.
If you want to see evidence of the campaigning and paid advertising that
candidates do, look to the movement in their favorable ratings. The table below
shows the most vivid evidence of the real progress that the candidates are
making in New Hampshire,
starting with the Democrats:
And the Republicans:
New analysis aggregated from the four most recent Gallup national surveys of 1,989 Democrats and those who lean Democratic (conducted 8/3 through 9/16) finds:
- Sen. Hillary Clinton leads Sen. Barack Obama (47% to 25%) in a national primary while former Sen. John Edwards trails at 13%. All other candidates average at less than five percent each.
- "Clinton holds a commanding lead among nearly every major subgroup of potential Democratic primary voters."
- A few "weak links" for Clinton: Among those in households with an annual income of $75k or more, she leads Obama 37% to 30%. Among Men aged 18-49 she narrowly leads Obama 39% to 34%.
A new University of Illinois at Springfield statewide survey of likely primary voters in Illinois (conducted 7/24 through 9/4; released 9/13) finds:
- Among 410 "very likely" Democratic primary voters in Illinois, Sen. Barack Obama leads Sen. Hillary Clinton (49% to 27%) in a statewide primary; former Sen. John Edwards trails at 6%.
- Among 286 "very likely" Republican primary voters, former Mayor Rudy Giuliani (at 31%) leads Sen. John McCain (18%), former Sen. Fred Thompson (15%), and former Gov. Mitt Romney (12%).
- All other candidates receive less than five percent each.
Two new Strategic Vision (R) statewide surveys of likely primary voters in Iowa and Florida (conducted 9/21 through 9/23) find:
- Among 600 Democrats in Iowa, Sen. Hillary Clinton (at 24%) narrowly leads former Sen. John Edwards (22%) and Sen. Barack Obama (21%) in a statewide caucus; Gov. Bill Richardson trails at 13%. Among 600 Republicans, former Gov. Mitt Romney leads former Mayor Rudy Giuliani (30% to 17%); former Sen. Fred Thompson trails at 13%, former Gov. Mike Huckabee at 8%, Sen. John McCain at 6%, and former Speaker Newt Gingrich at 5%.
- Among Democrats in Florida, Sen. Hillary Clinton leads Sen. Barack Obama (44% to 22%) in a statewide primary; former Sen. John Edwards trails at 12%, Gov. Bill Richardson at 6%. Among Republicans, former Mayor Rudy Giuliani leads former Sen. Fred Thompson (35% to 24%); former Gov. Mitt Romney trails at 9%, Sen. John McCain at 6%, former Gov. Mike Huckabee at 5%.
- All other candidates receive less than five percent each.
View all Iowa Caucus poll data at Pollster.com:
View all Florida Primary poll data at Pollster.com:
A new InsiderAdvantage/Majority Opinion statewide survey (results) of 743 likely voters in Kentucky (conducted 9/24 through 9/25) finds Democratic challenger Steve Beshear leading Republican Gov. Ernie Fletcher (45% to 35%) in a statewide gubernatorial general election match-up.
Kentucky Gubernatorial poll data:
Note: The InsiderAdvantage survey has consistently shown a higher undecided.
Additional results from the new CNN/WMUR Granite Poll (story, results) of 324 likely Republican primary voters in New Hampshire (conducted 9/17 through 9/24 by Univeristy of New Hampshire Survey Center) finds:
- Former Gov. Mitt Romney narrowly leads former Mayor Rudy Giuliani (23% to 22%) in a statewide primary; Sen. John McCain trails at 17%, former Sen. Fred Thompson at 12%, former Speaker Newt Gingrich at 7%. Without Gingrich, Romney runs at 25%, Giuliani at 24%, McCain at 18%, and Thompson at 13%. All other candidates receive less than five percent each.
- 13% have "definitely decided" who they will vote for in the New Hampshire primary, 21% are "leaning toward someone" and 66% are "still trying to decide."
- "[Romney's] statistically-insignificant, 1 point margin is a major change from CNN/WMUR's last New Hampshire poll, taken in July, when Romney held a comfortable 14 point lead over Giuliani."
Results for the Democratic Primary are available here.
View all New Hampshire Primary poll data at Pollster.com:
Via Jonathan Martin, a new Fabrizio, McLaughlin and Associates (R) national survey of 1,000 registered republicans (conducted 9/21 through 9/23) finds:
- Former Mayor Rudy Giuliani leads former Sen. Fred Thompson (23% to 18%) in a six-way national primary; Sen. John McCain trails at 12%, former Gov. Mitt Romney at 10%, former Speaker Newt Gingrich at 7%.
- 55% of Republicans say they would be "more likely" to support a candidate if he was endorsed by the president; 12% say "less likely," 31% say it would have "no affect."
Additional results from the most recent SurveyUSA automated survey (story; results) of 600 adults in New Mexico (conducted 9/14 through 9/16 for KOB-TV) finds:
- 41% approve of the job Sen. Pete Domenici is doing as U.S. Senator; 54% disapprove.
- "This is the lowest Domenici approval rating since SurveyUSA began tracking the senator's popularity in May 2005 and the first time his approval has sunk below 50 percent. His approval number has fallen from 68 percent last November and plunged 11 percent last month."
Carl Bialik, the Wall Street "Journal's Numbers Guy," warns
once again an online survey of online activity:
Would you be willing to give up online access for a
week? Then you might not be the most likely person to answer an online survey
(or to read this blog post, for that matter).
Yet a Web survey was the basis for articles by Reuters,
York Post and InformationWeek
claiming that 28% of Americans have let the Internet cut into time they spend
with friends and 20% have cut back on sex to spend more time online. Just 20%
of respondents said they could go without Web access for a week or more.
but it bears repeating: The people who answer online surveys aren't likely to
be representative of Americans when it comes to online behavior. They've found
or been found by survey companies, they've signed up to receive emails
announcing new surveys and they are among the first to respond to those emails
by filling out the surveys. Those are characteristics of people likely to use
the Web more regularly and devotedly than the average American.
I've seconded Carl on this point before.
post has more details and reactions from the company that conducted the
A new Quinnipiac University statewide survey of 1,230 registered voters in New Jersey (conducted 9/18 through 9/23) finds:
- Among 406 Democrats, Sen. Hillary Clinton leads Sen. Barack Obama (46% to 15%) in a statewide primary; former V.P. Al Gore trails at 11%, former Sen. John Edwards at 7%. All other candidates receive less than five percent each.
- Among 408 Republicans, former Mayor Rudy Giuliani leads former Sen. Fred Thompson (45% to 12%) in a statewide primary; Sen. John McCain trails at 8%, former Gov. Mitt Romney at 6%. All other candidates receive less than five percent each.
- General election match-ups:
Giuliani 45%, Clinton 44%
Giuliani 49%, Obama 40%
Giuliani 50%, Edwards 39%
Clinton 46%, McCain 41%
Obama 44%, McCain 41%
Edwards 44%, McCain 40%
Clinton 48%, Thompson 36%
Obama 49%, Thompson 34%
Edwards 48%, Thompson 34%
Clinton 52%, Romney 33%
Obama 51%, Romney 31%
Edwards 51%, Romney 30%
A new Rasmussen Reports automated survey of 1,200 likely voters nationwide (conducted 9/21 through 9/23) finds Gov. Bill Richardson edging out former Sen. Fred Thompson (42% to 41%) in a national general election match-ups while former Mayor Rudy Giuliani narrowly leads Richardson (43% to 40%).
A new CNN/WMUR statewide survey (story, results) of 307 likely Democratic
primary voters in New Hampshire (conducted 9/17
through 9/24 by the University of New Hampshire Survey
- Sen. Hillary Clinton leads Sen. Barack Obama (41% to 19%)
in a statewide primary; former Sen. John Edwards trails at
11%, former V.P. Al Gore at 7%, Gov. Bill Richardson at 6%.
Without Gore, Clinton runs at 43%, Obama at 20%, Edwards
at 12%, Richardson at 6%. All other candidates receive
less than five percent each.
- 17% say they have "definitely decided" who they will vote
for, 28% say they are "leaning toward someone" and 55% say
they are "still trying to decide."
- 93% are satisfied with the field of Democratic candidates;
8% are not satisfied.
Update: Full results are now available from University of New Hampshire.
Politico's Elizabeth Wilner
tracks the drop in Republican identification as measured by the NBC/Wall
Moines Register reports on a focus group of Republican caucus-goers
Read reports that Pollster Bill McInturff is leaving the McCain campaign.
Gallup Guru Frank Newport tells us ten
things we may not know about recent poll results and "unspins"
Hillary Clinton's recent Sunday talk show appearances.
The Corner's Kathryn
Jean Lopez cautions about reading too much into early general election
polls with the memory of some very wrong predictions from 1980.
Open Left's brklyngrl digs
deep into the combined
crosstabs released by the Cook Political Report/RT Strategies polls.
Boris Grushin, a pioneer of surveys in, yes, the Soviet
Union died Tuesday in Moscow
at age of 78 (New York
Times via AAPOR's member's only listserv).
And finally, the "read it all" item for this list: ABC's Gary
Langer answers criticism of the recent ABC/BBC/NHK survey
of Iraqi citizens and provides a detailed description of the unique poll
vetting operation at ABC News:
Years ago, with the support of
management, we set up a polling standards and vetting operation. First we
developed fair-minded but rigorous standards for what survey research we will
and will not report. And then a procedure by which any survey being considered
for air at ABC News goes through my unit first - or is supposed to - for a
review in which we check it out, and either clear it for air or kick it out the
A new Rasmussen Reports automated survey of likely primary voters in Florida (conducted 9/19 through 9/20) finds:
- Among 527 likely Democratic primary voters, Sen. Hillary Clinton leads Sen. Barack Obama (47% to 22%) in a statewide primary; former Sen. John Edwards trails at 11%.
- Among 508 likely Republican primary voters, former Mayor Rudy Giuliani leads former Sen. Fred Thompson (29% to 23%) in a statewide primary; Sen. John McCain trails at 12%, former Gov. Mitt Romney at 11%.
- All other candidates receive less than five percent each.
View all Florida Primary poll data at Pollster.com:
New analysis from the most recent Gallup national survey of 1,010 adults (conducted 9/14 through 9/16) finds:
- 53% rate the Democratic party favorably; 43% rate it unfavorably. 38% rate the Republican party favorably, 59% rate it unfavorably.
- 54% say the Democratic party "will do a better job of keeping the country prosperous;" 34% say the Republican party will.
- 47% say the Democratic party "will do a better job of protecting the country from international terrorism and military threats;" 42% say the Republican party will.
Four new SurveyUSA automated surveys in Virginia, New Mexico, Iowa, Missouri of about 500 registered voters in each state (conducted 9/14 through 9/16) finds:
Clinton 50%, Giuliani 44%
Clinton 50%, Thompson 43%
Clinton 53%, Romney 38%
Obama 46%, Giuliani 45%
Thompson 47%, Obama 45%
Obama 50%, Romney 38%
Edwards 48%, Giuliani 43%
Edwards 49%, Thompson 39%
Edwards 52%, Romney 33%
Giuliani 48%, Clinton 45%
Clinton 48%, Thompson 45%
Clinton 51%, Romney 40%
Obama 46%, Giuliani 44%
Obama 48%, Thompson 45%
Obama 51%, Romney 40%
Edwards 47%, Giuliani 42%
Edwards 50%, Thompson 40%
Edwards 56%, Romney 32%
Full crosstabs also available for Iowa and New Mexico.
A public memo circulated today by the Obama campaign and authored
by campaign manager David Plouffe (via Marc
Ambinder) argues that "Iowa is fundamentally a close three-way race with
Obama, Clinton and Edwards all within the same range in most public polling." His
characterization is reasonable, especially if one defines that "same range" as
roughly seven percentage points wide. Our own trend estimates for
Iowa based on all available public polls show both Clinton and Edwards
running a few points apart in the mid twenties (Clinton does slightly worse and
Edwards slightly better if we exclude
the polls by the American Research Group) with Obama trailing at roughly 20%.
But Plouffe goes on to make an assertion that is harder to
[P]olls consistently under-represent in Iowa,
and elsewhere, the strength of Barack's support among younger voters for at
least three reasons. In more than one survey, Barack's support among Iowa young voters
exceeded the support of all the other candidates combined. First, young voters
are dramatically less likely to have caucused or voted regularly in primaries
in the past, so pollsters heavily under-represent them. Second, young voters
are more mobile and are much less likely to be at home in the early evening and
thus less likely to be interviewed in any survey. Third, young voters are much
less likely to have a landline phone and much more likely to rely exclusively
upon cell phones, which are automatically excluded from phone surveys. So all
of these state and national surveys have and will continue to under-represent
Barack's core support – in effect, his hidden vote in each of these pivotal
early states. Of course, there are organizational challenges associated with
maximizing this support, but we are heavily focused on that task.
Each of the Plouffe's three arguments is at least
theoretically plausible, particularly
in Iowa, but
hard to prove or disprove conclusively with the data available.
Consider the cell phone effect. We know
that younger adults are much more likely to live in cell-phone only households,
that unweighted national poll samples
tend to skew older as a result but that age-related bias tends to fade to just
a percentage point or two (at most) when pollsters adjust their adult samples
to match census age estimates. However, in a state like Iowa, the big polling challenge is to select
the "likely caucus goers" that will hopefully represent the tiny sliver of adults
that will choose to participate in the caucus. The “census norms” available for
all adults are of much less utility when trying to determine the appropriate demographic
composition of the one-in-ten voters that we hope will represent likely
Pollsters will argue and disagree among themselves about the
best way to model and weight likely voters in a state like Iowa. We will not be able to resolve those
arguments here. What the rest of us should be looking for, at least, is whether
the various public polls are showing variation in their age composition and
whether any such variation is making a tangible difference in the results.
Although Plouffe may be cherry-picking an unusally favorable result, the national surveys consistently show Obama doing better among younger
voters. For example, in a combined
crosstabulation of its five most recent national polls (conducted since June),
the Cook Political Report/RT Strategies survey shows Obama
receiving 30% of the vote among 18-24 year olds, 24% among 25 to 49 year
olds and only 17% among those over 50. So if early state polls are under
representing younger voters, they may be slightly understating Obama's support.
But how much is the age difference in Iowa and how much do the Iowa
polls (or any of the other early states) vary by vary in their age composition? Who
knows? As far as I can tell, only the Time
survey conducted by in late August has reported its composition by age and
All of which brings me back to the Pollster.com
disclosure project. One of the most important reasons why we are requesting
additional details on the polls conducted in Iowa and the other early states is to allow
us all to better evaluate arguments like the one Obama's campaign manager made
today. So please read my post
from earlier today and comment or blog if you think this is a worthy idea. We would
appreciate your support.
Over the last few months I have written a series of posts that examined the remarkably limited methodological information released about pre-election
polls in the early presidential primary states (here, here
plus related items here). The gist is that these surveys often
show considerable variation in the types of "likely voters" they select yet
disclose little about the population they sample beyond the words "likely
voter." More often than not, the pollsters release next to nothing about how tightly
they screen or about the demographic composition of their primary voter samples.
Why do so many pollsters disclose so little? A few continue
to cite proprietary interests. Some release their data solely through their
media sponsors, which in the past limited the space or airtime available for methodological
details (limits now largely moot given the Internet sites now maintained by virtually
all media outlets and pollsters). And while none say so publicly, my sense is
that many withhold these details to avoid the nit-picking and second guessing that
inevitably comes from unhappy partisans hoping to discredit the results.
Do pollsters have an ethical obligation to report
methodological details about who they sampled? Absolutely (and more on that
below), and as we have learned, most will disclose these details on request as per the ethical codes of
the American Association for Public Opinion Research (AAPOR) and the National Council on Public
Polls (NCPP). Regular readers will know that
we have received prompt replies from many pollsters in response to such
requests (some pertinent examples here,
The problem with my occasional ad hoc requests is that they
arbitrarily single out particular pollsters, holding their work up to scrutiny (and
potential criticism) while letting others off the hook. My post
a few weeks back, for example, focused on results from Iowa polls conducted by the American
Research Group (ARG) that seemed contrary to other polls. Yet as one alert
I made no mention of a recent Zogby poll with results consistent with ARG. And while
tempting, speculating about details withheld from public view (as I did, incorrectly,
in the first ARG post), is even less fair to the pollsters and our readers.
So I have come to this conclusion: Starting today we will
begin to formally request answers to a limited but fundamental set of
methodological questions for every public poll asking about
the primary election released in, for now, a limited set of
states: Iowa, New Hampshire, South Carolina or for the nation as a whole.
We are starting today with requests emailed to the Iowa pollsters and will work
our way through the other early states and national polls over the next few
weeks, expanding to other states as our time and resources allow.
These are our questions:
the questions or procedures used to select or define likely voters or
likely caucus goers (essentially the same questions I asked of pollsters
just before the 2004 general election).
question that, as Gary Langer of ABC News puts
it, "anyone producing a poll of 'likely voters' should be prepared to
answer:" What share of the voting-age population do they represent? (The
specific information will vary from poll to poll; more details on that
will ask pollsters to provide the results to demographic questions and key
attributes measures among the likely primary voter samples. In other
words, what is the composition of each primary voter sample (or subgroup) in
terms of gender, age, race, etc.?
was the sample frame (random digit dial, registered voter list, listed
telephone directory, etc)? Did the sample frame include or exclude cell
was the mode of interview (telephone using live interviewers, telephone
using an automated, interactive voice response [IVR] methodology,
in-person, Internet, mail-in)?
- And in
the few instances where pollsters do not already provide it, what was the
verbatim text of the trial heat vote question or questions?
Our goal is to both collect this information and post it
alongside the survey results on our poll summary pages, as a regular ongoing
feature of Pollster.com. Obviously, some pollsters may choose to ignore some or all
of our requests, but if they do our summary table will show it. We are starting
with Iowa, followed by New
Carolina and the national surveys, in order to keep
this task manageable and to determine the feasibility of making such requests
for every survey we track.
Again, keep in mind that the ethical codes of the
professional organizations of survey researchers require that pollsters
adequately describe both the population they surveyed and the "sample frame"
used to sample it. The Code of
Ethics of the American Association for Public Opinion Research, for
example, lists "certain essential information" about a poll's methodology that
should be disclosed or made available whenever a survey report is released. The
relevant information includes:
The exact wording of questions asked . . . A
definition of the population under study, and a description of the sampling
frame used to identify this population . . . A description of the sample
design, giving a clear indication of the method by which the respondents were
selected by the researcher . . . Sample sizes and, where appropriate,
eligibility criteria [and] screening procedures.
The Principles of
Disclosure of the National Council on Public Polls (NCPP) and the Code of Standards and Ethics
of the Council of American Survey Research Organizations (CASRO) include very
similar disclosure requirements.
We should make it clear that we could ask many more
questions that might help assess the quality of the survey or help identify methodological
differences that might influence the results. We are not asking, for example, about response rates, the method used to
select respondents within each household, the degree to which the pollster
persists with follow-up calls to unavailable respondents or the time of the day
in which they conduct interviews. We have limited our requests to try to make
it easier for pollsters to respond while also focusing on the issues that seem
of greatest importance to the pre-primary polls.
What can you do? Frankly, we would appreciate your support. If
you have a blog, please post something about the Pollster Disclosure Project
and link back to this entry (and if you do, please send us an email so we can
keep a list of supportive blogs). If not, we would appreciate supportive
comments below. And of course, criticism or suggestions on what we might do
differently are also always welcome.
(After the jump - a more exhaustive list of the questions that
we will use to determine the percentage of the voting age population
represented by each sample)
Continue reading "The Pollster.com Disclosure Project"
Two new Rasmussen Reports automated surveys find:
- Among 800 likely voters nationwide (conducted 9/12 through 9/20), Sen. Barack Obama narrowly leads former Gov. Mitt Romney (46% to 43%) and Sen. John McCain (46% to 41%) in national general election match-ups.
- Among 500 likely voters in Georgia (conducted 9/12), Sen. Saxby Chambliss leads Democratic candidates Dale Caldwell (49% to 33%) and Vernon Jones (53% to 28%) in statewide general election match-ups for U.S. Senate.
A new American Research Group national survey of 1,100 adults (conducted 9/18 through 9/21) finds:
- 34% approve of the way George Bush is handling his job as president; 60% disapprove.
- 33% approve of the way Bush is handling the economy; 58% disapprove.
Via the Washington Post, a recently released Latino Policy Coalition survey of 1,200 likely general election voters in 31 "vulnerable" Congressional Districts currently held by Democrats (conducted 7/31 through 8/5 by Lake Research Partners (D) ) finds:
- 23% say "things in the country are going in the right direction;" 65% feel "they have gotten pretty seriously off on the wrong track."
- Among a half sample, former Mayor Rudy Giuliani leads Sen. Hillary Clinton (49% to 39%) and narrowly leads Sen. Barack Obama (41% to 40%) in general election match-ups.
- 51% would vote to re-elect their Democratic candidate for U.S. Congress (name given) if the election were held today; 32% would vote for the unnamed Republican.
- When prompted that the Democratic incumbent "is a strong supporter" of Clinton or Obama and will support their "liberal agenda of big government and higher taxes" and will be a "rubber stamp" and "forget the values of our district," the named Democratic candidate leads the unnamed Republican candidate by six percentage points.
Note: Lake Research Partners also polls on behalf of Democratic presidential candidate Joe Biden.