Mark Blumenthal | March 13, 2007
Topics: 2008 , The 2008 Race
Our friends over at the Hotline OnCall looked at the results of the latest CBS/NY Times survey and raised intriguing possibility:
Either early polls are fickle or Sen. John McCain is improving his standing vis-a-vis Rudy Giuliani, or both.
In the latest New York Times poll, Giuliani leads McCain by nine points, 43% to 34%. Last month, Giuliani led 50% to 29%.
Both the February and March surveys involved a hypothetical
two-way match-up between Giuliani and McCain (Q49),
and the decline in Giuliani's support appears to be just barely statistically
significant. For those wondering (as I did), surveys interviewed 314 registered
Republicans voters who say they are likely to vote in Republican primary in February and 603 in March (the most recent survey involved an oversample of
Republicans; unweighted counts are at the end of the CBS releases here and here).
Given the sample sizes, the seven point drop for Giuliani (from 50% to 43%) is just barely statistically significant at a 95% confidence level, although the five point increase for McCain (29% to 34%) is not.
However, there are some very good reasons for caution here. First, when I say "barely" I mean the Giuliani decline would not be significant if we needed to be 96% confident. And the precision of my calculation requires a statistical assumption (simple random sampling) that is iffy when pollsters weight their data (as this and virtually all other public polls do).
Next, remember that four other surveys conducted over the last 10 days - including the just released CNN/ORC survey - show no comparable Giuliani decline. In fact, while the changes on any individual poll may have been within sampling error, all four show a slightly larger Giuliani lead (see also the trend in evidence on our Presidential tracking chart).
Keep in mind, of course, that the CBS/NYT survey involved a hypothetical two-way Giuliani-McCain match-up, while the others asked respondents to choose among all of the likely Republican candidates. Notice that the CBS/NYT results are better for both Giuliani and McCain - the perfectly logical result of offering just two choices rather than roughly ten. Perhaps the Giuliani decline occurred mostly among those who only opted for Giuliani (in February) because their first choice was not an option.
Consider finally the issue of the "oversample" of Republicans on the most recent survey interviewed by redialing households "at which a self-identified Republican had been interviewed in [CBS/NYT] polls over the last seven months." Of course, the pollsters can compare the oversample to Republicans in the base sample to see if the former was more hostile to Giuliani. In response to my email query, CBS Survey Director Kathleen Frankovic confirms that she saw "no indication that the oversample was hostile to Giuliani."
When it comes to methodology, the CBS/New York Times surveys are top notch. But yes, early or late, even the best polls can produce "fickle" results. So given all of the above, readers should be cautious about making too much of the observed Giuliani decline until confirmed elsewhere.
UPDATE (3/14): Another possibility to consider is that CBS chooses it's likely primary voters a bit differently than the other national polls. For one, they actually attempt to identify likely primary voters, something not done by the other national polls (as far as I know). See this post today for more details. If the CBS/NYT poll is looking at a narrower population than other polls, they may be picking up emerging trends among true primary voters more readily.