June 29, 2008 - July 5, 2008


Two Weeks Worth of "Outliers!"

Topics: Barack Obama , John McCain

Desmoinesdem has some surprising advice for partisans on the receiving end of message testing polls they find offensive: don't hang up.

Kathy Frankovic ponders the potential for a perceived patriotism gap between McCain and Obama and reviews the problems that early voting creates for pollsters.

Frank Newport considers the value of pre-election polls in June, and lists five things you might not know about the election.

Gary Langer reviews misconceptions about the Hispanic vote.

Jennifer Agiesta reviews the Post/ABC numbers on McCain, Obama and Supreme Court appointments.

Kyle Dropp looks at public opinion on immigration.

Tom Schaller doubts Barack Obama can win in the South.

Nate Silver disagrees.

Election-Projection.net offers yet another election projection web site.

DemFromCt asks why the networks can't read polls.

Jay Cost considers Barack Obama's media buy strategy and his decision to forgo public funding.

David Hill doubts Barack Obama will convert GOP and independent youth and questions whether evangelical Christians will desert the Republicans.

Mark Mellman ponders what angers gun owners most, and addresses the "myth" that gay marriage initiatives determined the outcome of the 2004 election.

Jonathan Stein scores the Obama spreadsheet (via Bialik).

From last week, reviews on reaction to the Supreme Court's gun ownership decision from Jon Cohen, Gary Langer and Frank Newport,

And PPP polls the all-important topic of Andy Griffith and how his character might vote in 2008.

With that, we bid you a Happy 4th!

POLL: InsiderAdvantage Georgia

InsiderAdvantage/Majority Opinion
7/2/08 - 502 LV, 4.3%

Georgia (view our chart here)
McCain 46, Obama 44, Barr 4

POLL: DailyKos Connecticut

Research 2000/
DailyKos.com (D)
6/30 through 7/2/08; 600 LV 4%

Connecticut (view our chart here)
Obama 57, McCain 35

On Trackers

Topics: Barack Obama , Divergent Polls , Gallup , John McCain , National Journal , Pollsters , Rasmussen , USAToday Gallup

My Nationaljournal.com column, on the differences between rolling-average tracking polls and other "traditional" surveys, is now online.

Regular readers may be interested in the chart in the column created by our own Charles Franklin (see below) and spiffed up considerably by the National Journal's Reuben Dalke (see the column). I wondered how the tracking poll trends compare to standard trend estimates that you see on our national chart. The chart that Franklin created plots the trends on the Obama margin (Obama percentage minus McCain percentage) using a loess regression trend line based on the non-overlapping releases from Gallup Daily, Rasmussen Reports and all other national polls. To make for a fair comparison, all three lines are plotted with the same sensitivity.

I was also curious how the trends would look if we simply "connected the dots" between the non-overlapping tracking poll releases by Gallup and Rasmussen tracking surveys as well as the "traditional" USA Today/Gallup results (based on "likely voters") . You see that below.

The Gallup Daily line looks more variable than what you are used to seeing on Gallup's Daily release, partly because the time scale is more compressed, partly because we are plotting the Obama-McCain margin rather than separate lines for each candidate and partly because we are plotting only every third or fifth day which eliminates the "smoothing" effect of the overlapping intermediate samples.

What conclusions do you draw?

Update: In the comments, PatrickM asks:

As to the sampling process for the Gallup tracking survey: I thought the purpose of the tracking survey was to draw a discrete sample each night. Since completion quotas are set for each night, non-respondents must necessarily be "replaced" for that night's calls. Theoretically, all these replacements should balance out if non-response is random.

But Gallup seems to be taking a second bite at this apple by drawing an entirely new sample on the second night and supplementing it with non-respondents from the first night until the nightly completion quota is reached. So theoretically, the 3-day rolling results could include data from the originally drawn sample point AND its doppelganger replacement phone number.

I'm not a sampling expert, but is there anybody out there who can describe the rationale behind why this is OK?

The best explanation I have seen of "rolling cross section design" (a more technically correct term than "rolling average") is Kate Kenski's description of the National Annenberg Election Survey (NAES) in Chapter 4 in Romer, Kenski, Winneg et. al., Capturing Campaign Dynamics 2000 & 2004 .

The NAES, ongoing now for 2008 but mostly held back for academic analysis, uses the same general "tracking design" as Gallup only with far more rigor: In 2004, they protocol involved dialing non-contacts as many as 18 times over as many as 14 days.

I won't try to summarize the whole chapter, but this paragraph gets closest to answering Patrick's question:

What is important to note here is that there were strict procedures in place so that no telephone number was treated differently from any of the other numbers selected. Telephone numbers released on Tuesdays were not handled differently from telephone numbers released on Fridays. This protocol ensures that the probability of being interviewed is a random event. By stabilizing the proportion of respondents who completed an interview after having been called numerous times, the representativeness of the daily cross-sections is maximized.

Why is it important that the date of the interview be a random event? if the date of interview is random, then the characteristics of the sample on any given day will not vary systematically.

POLL: Rasmussen Montana

Rasmussen Reports
7/3/08 - 500 LV, 4.5

Obama 48, McCain 43

Comment of the Day

Topics: Barack Obama , Gallup , John McCain , NCPP , Sampling Error

From "joejoejoe," regarding yesterday's release of a new national survey from CNN/ORC:

Here's the headline to the CNN story that accompanies the poll.

'CNN poll: Obama, McCain in a statistical dead heat'

I'm not sure why a 5-point lead in a poll with a margin of error of +/- 3.5% is a "statistical dead heat" but whatever. Based on the '04 turnout a 1.5% victory projects to about 1.8 million more votes for Obama then for McCain. Doesn't "dead heat" mean tie?

Yes, it does, as as such "statistical dead heat" is a phrase we wish journalists would avoid.

To be fair, Nate Silver made the same point more emphatically (citing a National Council of Public Polls release) a few hours before joejoejoe. But we appreciate our alert readers nonetheless.

Update: A highly valued reader emails:

Not exactly. Suppose there are no undecideds and Obama leads 53-47. The +/- 3.5% MOE means that the estimate of 53% for the Obama has a 95% confidence interval ranging from 49.5% to 56.5%. 49.5% for Obama means 50.5% for McCain, so a McCain lead is within the margin of error. It's a little more complicated when there are undecideds, but the result would be similar

I probably posted this item too quickly. To me, and to most readers, "dead heat" means "tie." The point I agree with -- and the one made more directly by Nate Silver -- was not to imply that the 5 point margin was outside the margin of error, but rather to object to the use of the phrase "statistical tie" to describe a difference that is not quite large enough to attain statistical significance. It presumes we know the race is "a tie" when we lack the evidence, from this one poll, to be certain that a candidate is ahead.

Caution is always in order when it comes to interpreting small differences on just one poll result, but we have more than one poll to consider. Since May 1, we have logged 37 national poll releases (omitting daily tracking releases based on over-lapping samples). Only one (from Gallup) showed a "tie" result (44% to 44%). The other 36 had Obama ahead by margins of 1 to 15 percent. That's evidence that "tie" is not the best way to describe the current preferences in the race for president.

Poll: Daily Tracking for 7-02

Gallup Daily
6/29-7/1/08; 2,665 RV

Obama 46, McCain 44

Also from Gallup:
As Independents Shrink, Democrats Gain
Hispanic Voters Solidly Behind Obama (video)

Rasmussen Reports

6/29-7/1/08; 3,000 LV

Obama 49, McCain 44

To see how these numbers compare with the current trend, view our National Presidential chart here.

POLL: Quinnipiac University Connecticut

Quinnipiac University

n=2,437 likely voters, interviewed June 26-29

Obama 56%, McCain 35%

POLL: Strategic Vision Florida, Georgia

Topics: Incumbent Rule

The web site Political Wire published a sneak peak of at two new Strategic Vision (R) polls yesterday. While they included no information about survey dates and sample sizes, the reported the top-line numbers:

Florida: McCain 49%, Obama 41%, Barr 1%
Georgia: McCain 51%, Obama 43%, Barr 3%

We will update this entry (and our charts) with the results as soon we have the official release from Strategic Vision.

I corrected my typo and entered the numbers now posted by StrategicVision for Georgia and Florida. As of this writing the links for the releases appear to have the wrong release date (06/02/08) but list field dates of June 27-29 and a report different percentage for Obama in Florida (41%) than what PoliticalWire reported yesterday (43%).

POLL: CNN National

CNN/Opinion Research Corporation
6/26-29/08; 906 RV, 3.5%

Obama 50, McCain 45
Obama 46, McCain 43, Nader 6, Barr 3

To see how these numbers compare with the current trend, view our National Presidential chart here.

POLL: Rasmussen Massachusetts

Rasmussen Reports
6/30/08 - 500 LV, 4.5%

Obama 53, McCain 33
Sen: Kerry (D-i) 58, Beatty (R) 27

To see how these numbers compare with the current trends, view our Massachusetts Presidential (here) and Senate (here) charts.

POLL: Daily Tracking

Gallup Poll
6/28-30/08; 2,662 RV

Obama 47, McCain 42

Rasmussen Reports
6/28-30/08; 3,000 LV, 2%

Obama 49, McCain 44

To see how these numbers compare with the current trend, view our National Presidential chart here.

POLL: PPP Florida

Public Policy Polling (D)
6/26-29/08; 723 LV, 3.6%

Obama 46, McCain 44

To see how these numbers compare with the current trend, view our Florida Presidential chart here.

POLL: Southern Media & Opinion Research Louisiana

Southern Media & Opinion Research
6/26-28/08; 600 LV, 4%

McCain 52, Obama 36
Sen: Landrieu (D-i) 46, Kennedy (R) 40

To see how these numbers compare with the current trends, view our Louisiana Presidential (here) and Senate (here) charts.

POLL: SurveyUSA New York State

6/25-27/08; 531 LV, 4.3%

New York State
Obama 57, McCain 37

To see how these numbers compare with the current trend, view our New York State Presidential chart here.

Independent Leaners

Topics: Party Weighing

I neglected to link last week to a Washington Times op-ed by Gary Andres, a Republican who conducts and analyzes public policy polling projects for the lobbying firm, Dutko Worldwide, that examined the "swing voter universe." His column provided some helpful background on party identification and particularly the follow-up question asked of independents:

[T]here's an important distinction among so-called "independents." Many survey analysts ask a couple of follow-up questions. They ask those who say they are "Republican" or "Democrat" if their partisan affiliation is "strong" or "weak," and they ask independents whether they "lean" toward one party or the other.

These "leaners" are an important group. They represent a large chunk of the independents you read about in polls - in many cases as much as two-thirds of the group. But research has found these Americans are far from "independent." For example, those who "lean Democrat" vote for that party almost as consistently as partisans. The same pattern is true among independents who "lean Republican" - they vote heavily for the GOP. For example, in 2004, according to the American National Election Study poll, 83 percent of independents who "leaned" Democrat voted for John Kerry for president, just shy of his share among Democratic partisans. A similarly high percentage of "lean Republican" independents voted for George W. Bush. Both are a far cry from the conventional view of independents as an unpredictable "swing" group.

Including these "leaners" in the independent camp lumps in a lot of Americans who vote like partisans. That's why some surveys today even group these respondents in the Democratic or Republican camps when reporting partisan results. So, you might read a survey that reports the percentage of Republicans and Democrats "including leaners." That means the pollster has taken a group of people who initially might say they are "independent" and included them in a partisan category. Doing so, at least based on recent voting behavior, makes some sense.

There's more, and late link or not, it's worth reading in full.

Comment of the Day

Topics: Divergent Polls , Frank Newport , Gallup , LA Times/Bloomberg , Mike McDonald

From last Friday, posted by Michael McDonald:

[About the] Gallup Editors' Election Poll Analysis that attempts to place Gallup's tracking poll horserace numbers in context with other polls. Two of three contributions emphasize the "stability" of the Gallup poll, which is not something I find particularly compelling since random sampling error could produce the appearance of stability when there is instability.

Perhaps the most interesting explanation has to do with the question sequencing and wording, where Frank Newport takes issue with the LA Times/Bloomberg poll for asking the direction of the country question before the ballot test.

Still, I'd like to see Gallup pull apart their poll more. What are the demographics and support among groups in comparison to other polls? I think a beginning towards the answer why Gallup is different will lie somewhere in there.

I also find it a little disingenuous to explain the differences between Gallup and the two double-digit Obama lead polls only. I'm willing to believe the truth lies somewhere in the averages, but this begs the question why the Gallup daily tracking is different from the single-digit Obama lead polls.

POLL: PPP North Carolina

Public Policy Polling (D)
6/26-29/08; 1,048 LV, 3%

North Carolina
McCain 45, Obama 41, Barr 5
Sen: Dole (R-i) 51, Hagan (D) 37

Gov: Perdue (D) 42, McCrory (R) 41, Munger (L) 5

To see how these numbers compare with the current trends, view our North Carolina Presidential (here) , Senate (here), and Gubernatorial (here) charts.

The Hart/Annenberg Focus Group

Topics: Barack Obama , Clinton , Focus Groups , Washington Post

If you ever wanted to see a real political focus group from beginning to end (as opposed to the reality TV version that we sometimes see on cable news networks following a candidate debate), thanks to C-SPAN and the Annenberg Public Policy Center, you have a chance. The focus group sponsored by Annenberg and conducted last week in York, Pennsylvania by Democratic pollster Peter D. Hart is available for viewing from the C-SPAN web site, along with Hart's media debriefing. (both links are available from C-SPAN's main page if the foregoing links do not work) And if you would like to watch from the comfort of your sofa, C-SPAN will re-air the entire group tonight at 8:00 p.m. Eastern time.

The Hart/Annenberg focus group was also the subject of a must-read 2,200 word profile this morning by The Washington Post's Robert Kaiser. The success of this particular research tool, as Kaiser points out, "success depends on the skills of the person leading the discussion" and Hart is an especially skilled and experienced moderator. "A bad leader can ruin a focus group," Kaiser writes, "so can one or more ornery participants who try to dominate the proceedings." If you watch Hart carefully, you may notice he guides the group without imposing his own views, and as the evening progresses, gently coaxes the more dominant personalities in the room to hold their opinions for last.

Kaiser is also right that it is often "difficult to understand what is really important in a focus group discussion, and what is just noise." Here are some suggestions:

Be careful about counting. The issue is not so much size, as the difficulty of getting voters to participate in a process that requires in-person participation in a two-hour discussion group. Simply put, focus groups are not random samples, so we cannot use them to arrive at projective estimates of some larger population.

For example, Kaiser observes that "five of the seven Democrats who voted for Clinton in the primary were already comfortable with Obama as their candidate." That is an interesting finding, but it would be wrong to assume that 71% of Clinton supporters in York, Pennsylvania are now supporting Obama. The value is listening to those five individual explain their decision. We can understand the thought process and rationale of those five individuals, even if they do not produce "quantitative" estimates of a larger population of Clinton primary voters.

Who participated? The composition of the group frequently determines the impression that one gets from the discussion. One common philosophy among focus group researchers is to keep the composition as homogenous as possible. In the context of a political campaign, that usually means trying to limit participation to voters that are truly "persuadable" and still uncertain about their choice,

The York group included more "decided" voters than you might find in a typical focus group conducted by one of the campaigns, because in this case Hart and Annenberg wanted to understand how Obama is faring among Clinton's primary supporters. So they allowed primary voters to participate but screened out those who voted for Obama or initially preferred a Republican other than McCain. As a result, the group included quite a few primary voters who are now quite certain about their choice in November. I assume that their collective information level is also a bit higher than a group of truly persuadable voters (like the 23% of Americans described in this Gallup analysis) would be.

Pay attention to what isn't said. Much of Kaiser's piece, as well as the coverage from other journalists who watched from behind the glass, focused appropriately on what the participants said about Obama and McCain. However, I found this passage from Kaiser's article particularly telling:

The discussion, which lasted nearly 140 minutes, demonstrated again and again how little the participants felt they knew about Obama or McCain. "I don't know enough" was the substance of many answers to Hart's queries.

Consider it this way: Near the beginning of the group, Hart asked each participant to write down (and then describe) something that "has had an impact on me" in their thinking about either candidate in this election. Four mention McCain's recent proposal for off-shore oil drilling and three mention his "100 year" statement on Iraq. A few (5) more mention Michelle Obama's "proud of my country" remark and the controversial Reverend Wright (3). But think about the public policy issues that went unmentioned, either at that moment or later in the group. I heard, for example, not a word mentioned at any time about the candidate's positions on the FISA debate or Obama's decision to opt out of public funding for his campaign or a host of other policy issues that the candidates have spoken out on.

What gets said -- and what doesn't -- in this sort of open-ended discussion speaks volumes about the information that ultimately reaches voters in a campaign and what they do with it.

I am certain that our knowledgeable readers, some of whom are survey research professionals, can chime in with their own tips regarding focus groups and their interpretation. Our comments section is open as always. Have at it.

POLL: Rasmussen Florida, Alabama

Rasmussen Reports
6/26/08 - 500 LV, 4,5% (for each state)

McCain 48, Obama 41

McCain 51, Obama 36
Sen: Sessions (R-i) 58, Figures (D) 34

POLL: SurveyUSA Massachusetts

6/25-27/08 - 607 LV, 4.1%

Obama 53, McCain 40

Why So Much Variation? (Redux)

Topics: Divergent Polls , Gallup , LA Times , Measurement , Newsweek , Party Weighing , Pew Research Center , Pollster.com , Rasmussen

There has been quite a bit of speculation this week over that perennial we enjoy here at Pollster.com: Why are the polls so divergent? In the last week or so, we have had national polls showing everything from a dead even race (Gallup Daily) to a 15 point Obama lead (Newsweek) and a half-dozen or so results in between.

Here are three more sets of reactions and analyses from pollsters themselves:

  • Los Angeles Times pollster Susan Pinkus, as quoted in a blog item from the Times' Don Frederick
  • Richard Morin, now of the Pew Research Center, formerly polling director for The Washington Post, as interviewed by CQ Politics.

Cumulatively, this discussion covers the usual suspects: differences in party identification as measured by the surveys, as well differences in question wording and question order. I wrote about the party ID debate in my column last week, something that Pinkus, Morin and Rasmussen address at the links above.

The Gallup editors raise the issue of question wording and order. Frank Newport points out the Gallup Daily begins with the vote question, while the L.A. Times poll started with the standard question about the direction of the country. He also points out that the L.A. Times vote question includes the phrase, "or would you vote for a candidate from some other party," an option not offered explicitly by the Gallup question.

The decision of the L.A. Times pollsters to ask the vote question second, after the "right-direction, wrong track" question, struck Nate Silver as a "big no-no." It is true that most of the media pollsters have now shifted to asking the presidential vote question at the very beginning of the survey, a practice they generally follow in surveys in the last few months before the election.

But not all do. Most of the pollsters doing internal "benchmark" surveys for campaigns take a different approach that usually involves asking questions about the direction of the country and favorable ratings of the candidates before asking about vote preference. You can see examples of this philosophy in the survey out today from Democracy Corps poll conducted by Democratic firm Greenberg-Quinlan-Rosner, or the most recent NBC/Wall Street Journal poll.

The rationale for that question order is that voters do not make their choices on a whim. They cast their ballots after some reflection at the end of the campaign, not by being ambushed and asked to make an immediate decision in the middle of the summer. If neutral questions that get respondents thinking about the candidates and the direction of the country for a minute or two produce a result different than asking immediately for vote preference, then the effect gets us closer to the way they will approach the real campaign. As Silver put it in a second thought, posted after some reflection:

If the mere suggestion that the country might be on the wrong track is enough to send scores of independents into the Obama column, imagine what a concerted effort to frame the discourse that way might do.

It is always important to consider question wording and question order, although I'm dubious that the placement of the "right direction wrong track" question alone explains why the L.A. Times had a better result for Obama than most other recent polls. After all, the Newsweek poll started with the vote questions, and they showed an even bigger (15 point ) margin for Obama.

POLL: Democracy Corps National

Democracy Corps (D)/
Greenberg Quinlan Rosner (D)
6/22-25/08 - 2,000 LV

Obama 49, McCain 45

Generic Ballot
Pres: Dem 50, Rep 43
House: Dem 50, Rep 42

George W. Bush
Job: Approve 34, Disapprove 61

(Split - 500 Respondents) "Given the high price of gas, would you favor or oppose allowing oil drilling in U.S. coastal areas and wilderness areas that are currently protected by the federal government?"

    56 Favor
    39 Oppose

(Split - 500 Respondents) "As you may have heard, the Republican Nominee, John McCain has proposed allowing oil drilling in some U.S. coastal regions to help lower the price of gas. Do you favor or oppose this proposal?"

    61 Favor
    35 Oppose

POLL: Elway Washington

Elway Poll
6/18-22/08 - 405 RV, 4.9%

Gov: Gregoire (D-i) 47, Rossi (R) 39

POLL: SurveyUSA Virginia

6/20-22/08 - 630 LV, 4%

Obama 49%, McCain 47%

POLL: Rasmussen Georgia

Rasmussen Reports
6/26/08 - 500 LV, 4.5%

McCain 53, Obama 43, Barr 1
Sen: Chambliss (R-i) > 50, Dems < 40