Pollster.com

Articles and Analysis

 

Re: 46-45 Plus or Minus 3

Topics: 2008 , Barack Obama , Divergent Polls , Frank Newport , Gallup , Hillary Clinton , Likely Voters , Rasmussen , Sampling Error

Update: In the comments, Chris G argues that I am "way off" to conclude that "there has been far more stability than change in the national Obama-Clinton vote preference since Super Tuesday." He writes:

[T]hat simply does not follow from the simulations. the only thing that can be inferred is that if we're looking at these 2 time series alone, any meaningful changes in support are swamped by the noise. that's all we can conclude.

Since I may have been unclear, let me try to clarify: I am not arguing that the Gallup Daily and Rasmussen Reports tracking data proof the complete absence of change in candidate preference since Super Tuesday. Chris is absolutely right: No survey can do that. The best we can do is conclude that changes have been too small to be detected with confidence.

The point I was trying to make is that the changes since Super Tuesday have been (a) short lived, (b) small enough that they are indistinguishable from random noise, or (c) both. I do not consider changes of that sort to be very meaningful substantively, though your definition of "meaningful" may differ.

I am also not arguing that we should ignore the Gallup Daily. We just need to be patient and wait to see big, persistent changes. Look back at the numbers they reported in January through early February and you can see a very large, sustained and meaningful trend toward Obama:

03-25_GallupDaily.jpg

In the midst of writing this update, I discovered that Gallup's Frank Newport made essentially the same point in his daily video report today:

As I look at the Gallup Daily election tracking, I am struck by the fact that neither candidate, Hillary Clinton or Barack Obama has been able to move ahead to a sustained and significant lead over the other [emphasis added].

 

Comments
Joseph E:

Perhaps I wasn't paying attention at the time, but in retrospect one of the most striking changes happened when Edwards dropped out.

Prior to the end of February, Obama was stuck at about 30 to 35%, while Clinton had over 40 to over 45%, with Edwards taking over 15%.

When he dropped out, Clinton's numbers in the Gallup Daily went up about 5%, but Obama picked up 10%. It looks like Obama also got about 5% from Clinton and undecided voters around that time (and lost a little right before Super Tuesday, again), and some of that could be random noise.

But I think it is clear that a majority of Edward's supporters went to Obama, and that this was significant in his winning the pledged delegate lead in early February. Before, I heard as much speculation toward the other side (arguing that Edwards was splitting white or older or traditional voters with Clinton, rather than taking Change votes from Obama)

____________________

You are spot on, and that stability is why I decided that this was over after Wisconsin and I'm sticking to that until shown otherwise. Nothing has budged this any significant amount.

Why would that the be case? I should warn you, I'm a man with a theory ... and that theory is that this election is really all about generational change. On my blog, link above, I've been going over the changes I expect as people born after 1965 move to between 37 and 45 percent of the overall electorate for the first time this year.

I'd like to see a lot more polling analysis by age. While the old stand-bys of gender and race show patterns, my hunch is that they will be even strong when you look at generational identity. That's not something which has changed since Super Tuesday, and the polls show the stability of identity.

____________________

Alex:

re: Joseph E

I actually think both effects were true, and the way I interpret the split was that when Edwards dropped out, a group of them quickly changed to Obama supporters, while another group became undecided and slowly chose Clinton over a slightly longer period. It still looks about 2 to 1 Obama, as there's about a 10 point increase for Obama and a 5 point for Clinton after Edwards left the race.

Also, I think there is probably a contingent of Edwards supports (very small but slightly significant) that went to Huckabee or McCain. I remember that in South Carolina exit polls, Edwards was winning the conservative, pro-war Democrats and a good chuck of older white men. I think the former group includes the Lou Dobbs populists who could easily jump between Huckabee and Edwards or even Paul, and the latter group being ones who were skeptical of the country's ability to overcome racial and gender patterns. I think Edwards ended up being a niche in a place he didn't want to be. I could see some of those folks going back to the Dems in the general regardless of the candidates, due to concerns about the economy or the war.

So I agree with you that Edwards dropping out was a significant factor in evening out the support of the two remaining candidates.

____________________

cinnamonape:

I actually don't see much gain for Clinton from Edwards withdrawal at all. Given that Rassmussen does almost daily, three day rolling polls we might expect one or two 4-5 point outliers in the series. Thus a 50% singular poll might be expected from an average of 45%. It seems that Clinton has essentially stayed put...she hasn't convinced anyone more than she already had since January.

If one did want to argue the reality of the little peak that occurred after Edwards dropped out, I'd attribute it just as much to the approach of Super Tuesday which compelled "undecideds" to make a choice. The exit polls have consistently shown that later deciders tended to swing a bit more for Clinton. After Super Tuesday they went back to being "undecided". That's only if you believe the blip wasn't a statistical outlier, though.

The "Edwards factor" seems much more salient for Obama. He won over individuals that were pro-Edwards disproportionately (and I'd say at higher than a 2:1 ratio). There may have also have been a bit of the "Super Tuesday effect" compelling undecideds to make up their minds. But either they have stuck with him, or their movement back to the "undecided" ranks has been balanced by others won over since.

____________________

cinnamonape:

I actually don't see much gain for Clinton from Edwards withdrawal at all. Given that Rassmussen does almost daily, three day rolling polls we might expect one or two 4-5 point outliers in the series. Thus a 50% singular poll might be expected from an average of 45%. It seems that Clinton has essentially stayed put...she hasn't convinced anyone more than she already had since January.

If one did want to argue the reality of the little peak that occurred after Edwards dropped out, I'd attribute it just as much to the approach of Super Tuesday which compelled "undecideds" to make a choice. The exit polls have consistently shown that later deciders tended to swing a bit more for Clinton. After Super Tuesday they went back to being "undecided". That's only if you believe the blip wasn't a statistical outlier, though.

The "Edwards factor" seems much more salient for Obama. He won over individuals that were pro-Edwards disproportionately (and I'd say at higher than a 2:1 ratio). There may have also have been a bit of the "Super Tuesday effect" compelling undecideds to make up their minds. But either they have stuck with him, or their movement back to the "undecided" ranks has been balanced by others won over since.

____________________

Chris G:

Mark - I appreciate your response, I think it's important to step back and consider that there're far more polls out there than Gallup and Rasmussen. by including more polls, Dr Franklin's local regressions reduce error and give us a clearer view (although methods do differ across polls)

here's another reason why I think this is important: Clinton's "kitchen sink" strategy has given us a preview of things to come in the general, and the local regression suggests that in the past 4 weeks Clinton's support may have risen by a small amount. I personally think it's unlikely she'll ever catch up in the popular vote. but Obama's challenge is to figure out how to fend off similar attacks come the fall.

so again, my main point is that we shouldn't just write off dynamics that happen to be too small to detect in a single time series. if we go back to the 2000 election, many talked about "volatility" in the numbers although I think much of this was also noise. nonetheless, at the end of the day every % point counted

so I'll punt back to you: how do we detect and measure changes in electoral support that are too small for a single poll to pick up over time, but may nonetheless be decisive come election day?

____________________

JS:

Mark,


Thanks for you response to my post yesterday.

I am not sure I was clear in making my point, which was a technical statistical but important one.

You original post said that the MoE applies to the different percentages, not to the difference between them.

I debate that. The MoE of the difference in percentages is about the same as each individual percentage, even in the same sample. (The SE, on which the MoE is based, is slightly different. The the sample SD which is the estimator, is slightly different since the percentages are different, but only slightly; and you lose one degree of freedom.).

Look at this example. If you test Ho: X (Obama ) = 0 the SE of the sampling distribution would be = SE (X (Obama)). If you test Ho: X (Clinton ) = 0 the SE of the sampling distribution would be SE (X (Clinton). And, SE (X (Obama)) = SE (X (Clinton)), or more precisely, is mathematically close. (See above.)

Further, and this is my point: If you test Ho: X(Obama) - X(Clinton) = 0, the SE is about the same as either of the two individual tests.

Therefore, the MoE for comparing Clinton and Obama's percentages is not double the MoE for each individually. It is a separate calculation, which yields, in this case, an almost identical MoE.

As to your point that it cannot be argued that these are separate samples, I agree. They are the sample sample. Nevertheless, a t-test for a difference of means in the same sample (as opposed to a paired difference of means t-test), is about identical to the test for each mean individually.

I would be happy to send the equations, but it is hard to do in text.

This seems to be a common error in evaluating the difference in two candidates poll results. Typically, as I believe you did, the argument is that each percentages' outer-bound MoE must lie outside the outer-bound of the other's MoE. In effect doubling the MoE to get a statistically significant result. What should be done is to calculate the MoE of the difference in percentages, which is base on a SE distribution which will be almost identical to each SE individually.

Again, the practical implication is that there are more statistically significant differences than you are reporting by your method.

This is very dry statistics, but the resulting point is important.

Perhaps I am missing something in your argument. Let me know. I am interested in your evaluation of the above.

And thanks for your work.

JS

____________________

Mark Lindeman:

JS: I don't know what Mark B. thinks about your analysis, but it seems to me that your "example" becomes more nearly an assertion at the crucial point. And it seems to me that the results from Polimatic actually demonstrate that because Obama's and Clinton's vote shares aren't independent, the SE of the difference is substantially larger than you think.

It's true that people often add MoEs (or SEs) when they shouldn't. If the two means are independent, then the SE of the difference should be about sqrt(2) times either individual SE (analogous to a sum of two orthogonal vectors). Unless I'm missing your point completely, which is certainly possible at this time of night.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR