Pollster.com

Articles and Analysis

 

Zogby Internet Poll Trial Heats are Odd

Topics: Divergent Polls

1TrialHeatsZogbyHC1126.png

A new Zogby Interactive poll, conducted using volunteers over the internet, has produced some odd results for trial heats involving Senator Clinton against all four top Republican opponents. What makes this especially odd is that the results are not equally unusual for Obama.

This poll was reported by Reuters' John Whitesides, who also reports on the Reuters sponsored polling Zogby does by conventional telephone methods. The similarities in the reports make it hard to tell, but apparently these results are not part of the Reuters-Zogby polling partnership, but are independent work by Zogby Interactive. Likewise Zogby's website posts the results without mention of who sponsored the work, so presumably Reuters did not.

The Zogby poll was conducted 11/21-26/07 with 9150 respondents who had agreed to take part in Zogby's online polling. This is not a normal random sample of the population. More on the technical issues below.

The hugely surprising result is that the Zogby poll finds Sen. Hillary Clinton losing to all four top Republicans in head-to-head trial heats. What makes that surprising is that Clinton LEADS all four of those Republicans in the trend estimates based on all other polling by between 3.8 and 11.6 points. Zogby also has Clinton losing to Arkansas Governor Mike Huckabee by 5 points. There are too few Clinton-Huckabee trial heat polls from other organizations for me to compute a trend estimate for that comparison.

The chart above shows all the trial heat data from national polling and the estimated trend lines for each pairing. The data points for the new Zogby data are indicated in the charts as "Zogby Inet" in blue for Clinton and red for each Republican.

What is immediately clear is that the Zogby Clinton numbers are well below the estimated trend for Clinton in each of the four comparisons. Clinton is consistently 8-10 points below her trend estimate based on other polling.

In contrast, the Republican results are quite close to the trend estimate in most cases: Giuliani is at 43 in Zogby, with a trend of 44. Romney is 43 in Zogby, 38.3 in trend; Thompson is 44 in Zogby, 41.3 trend, and McCain is 42 Zogby, 42.7 trend. Those Republican numbers are about the kind of normal noise we see around the trend estimate, so don't seem out of line.

Why then is Clinton so far down in comparison to other polls? The Reuters story doesn't note that these results are far from other polling, and instead uses the theme that Clinton is declining to frame these Zogby results:

The results come as other national polls show the race for the Democratic nomination tightening five weeks before the first contest in Iowa, which kicks off the state-by-state nomination battles in each party.

Some Democrats have expressed concerns about the former first lady's electability in a race against Republicans. The survey showed Clinton not performing as well as Obama and Edwards among independents and younger voters, pollster John Zogby said.

While this is certainly a theme of recent reporting, boosted by a pre-Thanksgiving ABC/WP poll showing Obama leading Clinton in Iowa, it is striking that no other poll has found recent results as far from the trend estimates as are Zogby's results and that the Reuters story fails to note that fact.

One answer to why Clinton does so badly MIGHT be that the poll has too few Democrats and thus biases its results. But if that were so, we'd expect Obama to also underperform his trend estimates. That doesn't happen, as the chart below makes clear.

2TrialHeatsZogbyBO1126.png

The Zogby results for Obama are all quite close to his trend estimate from all polls:

Zogby has Obama at 46% vs Giuliani, while the trend puts him at 44.3. Against Romney Zogby has Obama at 46%, while trend says 46.6. Against Thompson Zogby has Obama at 47, while trend is 47.0, and against McCain Zogby has Obama at 45 while trend puts him at 43.4.

This is clearly not consistent with a general anti-Democratic bias in the Zogby Internet poll. It is also clear from the graph that the Obama pairings find Republicans doing quite close to the trend estimates as they did against Clinton.

(Trial heats against Edwards are not very common recently, so the Zogby results for him lack much polling for comparison.)

And so we are left with a puzzle: What is it about these respondents that so strongly affects Clinton support but no one else?

We can probably rule out one easy explanation: That Clinton has suddenly collapsed and Zogby is just the first to find it. The reason is internal to the Zogby result. If Clinton really has suddenly become 10 points less attractive, we'd expect all four Republicans paired against her to do BETTER than their trend estimates when facing her. But what happens is Clinton goes down and they don't do any better. That is hard to reconcile with a real change in Clinton's support. (A tortured version would say Clinton must have collapsed among Dems who now say they are undecided while refusing to move towards any of the Republicans. But that isn't usually what happens in real data when one candidate declines sharply. Usually the other moves up at least a bit, drawing not only from unhappy partisans but especially from independents who now are disenchanted with the former front-runner. So while you could make the math work with this story, it doesn't seem very well supported by the data.)

The Zogby Internet polling has a questionable track record in statewide races for Senate and Governor in 2006, where they often far over-estimated the competitiveness of races compared to conventional phone polls taken at the same time. One way to make sense of those problems turns out not to help much here. It is reasonable that the people who volunteer to take political polls over the internet are considerably more interested in politics (and likely more strongly partisan) than is a random sample of likely voters. That should be expected to lead to fewer people with "don't know" responses as better informed and more partisan respondents are likely to both know more about the candidates and to have made up their minds sooner than a proper random sample. That helps explain why Zogby's 2006 internet polls looked as they did.

But this does no good in Clinton's case. What we see is that MORE internet respondents are undecided about their vote between Clinton and four Republicans than the trend estimates based on less involved and partisan phone samples show. The Zogby undecided rates for the Clinton pairings are 20, 17, 17 and 16% (plus 17% undecided in the Huckabee comparison.) The comparable undecided rates based on the trend estimates are 8.2, 12.8, 9.0 and 10.6. That is an average undecided rate of 17.5 in Zogby vs 10.15 in the trends. Likewise the undecided rate is slightly lower for Obama pairings than it is for Clinton: 17, 13, 14, 13, and 14 for Huckabee. How could it be that a sample that is almost certainly more involved, knowledgeable and partisan can be LESS decided about Cinton, the single best known figure in the race? Again, a tortured story might be constructed, but I think a simpler explanation is that this result is not consistent within the Zogby data itself, or in comparison with outside polling.

Where does this leave us? Puzzled. If these results came from voting machines, I'd suspect that something in the ballot design or the recording mechanism caused a modest but consistent undercount of the Clinton support. The effect seems confined only to that one candidate, and not to any others, Democrats or Republicans. And there was no boost in support for the Republicans paired against Clinton. In this case, I'm similarly inclined to wonder if there is the possibility that the Zogby online survey had a glitch that caused a systematic "undervote" for Clinton. Certainly if my research assistant brought me these results, I'd want to check the software for mistakes before I published it.

Let's assume the Zogby organization has checked for any such possible mistakes or glitches and has ruled that out. (One would assume they were as surprised by the data as anyone and since their reputation is on the line, would have checked very carefully before releasing the data.) Is there any reasonable model of how candidate preferences are evolving that might explain this result, and the stability of Republicans paired against Clinton AND the stability of Obama support and that of his Republican pairings?

Without access to the raw data it is impossible to test any speculation here. But here is one possibility: Internet polls, presumably including Zogby's, use weighting to adjust for non-representativeness in their volunteer respondents. (There is a huge debate about whether this, and more sophisticated approaches, can produce generalizable population estimates with good statistical properties, but we'll leave that for another day.) Clinton has more support among women and somewhat older people. Both those groups are likely to be underrepresented in any pool of internet respondents. As a result the responses of those with these characteristics who ARE present in the sample are likely to be weighted up quite a bit to reach population proportions in the weighted sample. If the relatively few older women who are in the sample are ALSO atypical in other ways that both make them volunteer for internet surveys AND be less disposed to support Clinton than are non-internet volunteering older women, then weighting these respondents up won't properly capture Clinton's support and will lead to a systematic underestimate of her support.

That could do it, but it sounds pretty tortured to me.

I'd check the software one more time.

And based on the large outliers the Clinton results produce, I'd hold off on the Reuters headline until I saw some confirmation from other polls.

Cross-posted at Political Arithmetik.

 

Comments
dcshungu:
Zogby Internet Poll Trial Heats are Odd
Pollster.com addresses the concern expressed in an Open Letter to them and Zogby International by dcshungu in a Talking Points Memo blog's Election Central thread on November 26, 2007 5:15 PM.

However, the response fails to state the obvious: Clinton loses to all the GOP candidates because the Zogby "interactive" internet poll's methodology is seriously flawed. The poll was overwhelmingly taken by internet-savvy "passionate" Clinton detractors, looking for some relief from their Clinton Derangement Syndrome (CDS), who repeatedly (figuratively, of course) pressed the anti-Clinton virtual lever when provided with the opportunity. This type of relief is analogous to that experienced by lab mice that are conditioned to repeatedly press some "lever" for reward...

The evidence: Please explain how Clinton could possibly be trailing all the Republican candidates nationally, including Romney and Huckabee, who remain virtually unknown outside of the few early primary or caucus states where they have campaigned...

This poll is not scientific. I know a thing or two about stats and the "scientific method." Please do the electorate and democracy a favor and avoid posting such results, lest you do great damage to your firm's reputation and credibility.

DCShungu, Ph.D.
Associate Prof of Physics in Radiology,
Psychiatry, Physiology and Biophysics.
@ a couple of Ivy League Medical Schools in Manhattan.

____________________

This is just a biased poll, either by design or by poor planning. I've heard enough about Zogby International to not trust their telephone polls over other companies, I certainly don't trust their internet polls.

____________________

Nick Panagakis:


If I read your post correctly, Clinton is the only Democrat who does not do well against GOP opponents. And, she does not do as well as she is in other polls.

My first reaction is that the gender weight was off. M/F ratio should be about 52%/48%, perhaps 53%/47%.

As we all know, Clinton's support comes disproportionately from women.

Is there any way to check on gender?

____________________

DTM:

dcshungu,

Rather than not reporting on polls using alternative methods, the people at pollster have made a practice of reporting on them but also discussing their methodology, comparing their results to more traditional polls, tracking their accuracy in different contexts, and so on.

I for one appreciate pollster's approach because at a minimum it maximizes the information available to us. I would also suggest it is a quintessentially "scientific" approach, in that rather than prejudging the accuracy or lack thereof of these alternative polling methods, pollster is evaluating these issues empirically.

____________________

JohnG:

I would chalk this one up to simply randomness. One poll does not make a trend, and the sample was small enough that a few responses might have been enough to throw it off. Suppose, for example, that the sample included a greater than average number of "Clinton haters" - that would account for her doing poorly.

____________________

dcshungu:
Nick Panagakis wrote:

My first reaction is that the gender weight was off. M/F ratio should be about 52%/48%, perhaps 53%/47%.

As we all know, Clinton's support comes disproportionately from women.

Is there any way to check on gender?

I do not think that would reveal much. The correct "control" experiment would be to determine the relative political leanings of the respondants, which can be achieved simply by having them (republicans and democrats) express their preference in a poll that would pit the Democratic candidates against each other. I suspect that an ideologically "pure" candidate like Dennis Kucinich would do very well among voters in this sample who consider themselves Democrats, but overall, I would not be surprised to see even Kucinich beating Clinton using the methodoogy of this poll. However, for many in the sample, the votes be won't for Kucinich as much as they would be votes against Clinton. What I am driving at is that this sample would turn out to consist primarily of Republican voters who think that Clinton is too "liberal" and Democratic voters who think that she is "conservative" enough to be a Republican Fifth Column among the Democrats. In other words, Clinton is too centrist for them, and fortunately for her, she is ideologically where elections are won because the ~20% or so "swing voters" who usually decide elections are mostly near the center of the American political divide!

____________________

dcshungu:

DTM said:


I for one appreciate pollster's approach because at a minimum it maximizes the information available to us. I would also suggest it is a quintessentially "scientific" approach, in that rather than prejudging the accuracy or lack thereof of these alternative polling methods, pollster is evaluating these issues empirically.

It is not that we are short of polling data! If anyone intends to use a flawed polling methodology for didactic purposes, s/he should present the flawed poll and discuss its flaws at the same time. As you know, polling results can drive or alter the "narrative" of a campaign. Thus, to report polling results which are known to be flawed and let spread the impression that the numbers are meaningful could alter the dynamics of an election in a way that might serve one candidate or another, but would serve neither the public nor democracy very well.

Since in no branch of "science" would a study with flawed stats see the light of day, why should this be tolerated in political "science" where polls can make or break campaigns, drive the debate or shape public policy?

Using flawed polling methodologies for didactic purposes is fine as long as it is made absolutely clear from the git-go that the poll numbers are meaningless because the sampling method is flawed...

____________________

Mark Lindeman:

Jeepers. Studies with flawed stats see the light of day all the time, especially when politics is involved. Let's keep it real around here.

In this case, pollster.com flagged the data issue immediately and followed with a detailed analysis a day later. In contrast, Reuters reported the results as straight news, not even mentioning that the survey was an online poll until the very end (although cognoscenti familiar with "Zogby Interactive" would have figured it out). I don't know who would be better off if pollster.com had ignored the poll.

____________________

BDM:

Possible explanations for Zogby:

1. DATE POLL TAKEN nOV. 21-26, Gallup taken Nov. 11-14th

2. Rasmussen poll taken Nov. 19-20th seems consistent with Zogby:

Survey of 800 Likely Voters
November 19-20, 2007

Rudy Giuliani (R) vs.
Hillary Clinton (D)

Rudy Giuliani (R)
46%

Hillary Clinton (D)
42%

Hillary Clinton (D) vs.
Fred Thompson (R)

Hillary Clinton (D)
46%

Fred Thompson (R)
44%

3. Clinton has received negative new's coverage from mafter her debate performance of OCT. 30TH. tHIS WAS A 9 DAY STORY. aLSO THE ABC Poll was released before Zogby's polling began and got a lot of press.

Finally the narrative of the MSM changed after the debate of OCT. 30TH AND IS NOW STARTING TO BE PICKED UP IN POLLING.

OR

This poll is an outlier.

____________________

gujarbhai:

Gallup Poll on Nov 26 follows the General trend.

http://www.gallup.com/poll/102862/Democratic-Candidates-Look-Good-Latest-2008-Trial-Heats.aspx

*
Pres '08
Nov 26 Gallup
Clinton (D) 49%, Giuliani (R) 44%
*
Pres '08
Nov 26 Gallup
Clinton (D) 50%, McCain (R) 44%
*
Pres '08
Nov 26 Gallup
Clinton (D) 54%, Romney (R) 38%
*
Pres '08
Nov 26 Gallup
Clinton (D) 53%, Thompson (R) 40%
*
Pres '08
Nov 26 Gallup
Obama (D) 45%, Giuliani (R) 45%
*
Pres '08
Nov 26 Gallup
Obama (D) 47%, McCain (R) 44%
*
Pres '08
Nov 26 Gallup
Obama (D) 52%, Romney (R) 35%
*
Pres '08
Nov 26 Gallup
Obama (D) 51%, Thompson (R) 38%

____________________

BDM:

That Gallup poll released on Nov. 26th was taken on Nov. 11-14th. My point is that Zogby and Rasmussen's poll's were taken nov. 21-26th for Zogby AND nOV. 19-20TH for Rasmussen.

____________________

dcshungu:
the problem is there are no really scientific polls because it's very difficult to get a really random set and many people refuse to be in polls (I think I've heard 40% is typical). This means all polls are adjusted to try to make the group like the general population, so, for example, if there are too few women in the sample they will be counted more to make up for it.
The idea behind Zogby's internet poll is to get a large group of people who will agree to answer any poll and then get lots of info about them, so the results can be adjusted. This means that if something has changed it will not catch that change (for example, the 1936 election was one of the first where class made a large difference in whether a person voted Dem of Rep).
I still trust 'random' samples more, but it's not as clear as theory would state. Therefore, we should see how the two types compare.

____________________

dcshungu:
the problem is there are no really scientific polls because it's very difficult to get a really random set and many people refuse to be in polls (I think I've heard 40% is typical). This means all polls are adjusted to try to make the group like the general population, so, for example, if there are too few women in the sample they will be counted more to make up for it.
The idea behind Zogby's internet poll is to get a large group of people who will agree to answer any poll and then get lots of info about them, so the results can be adjusted. This means that if something has changed it will not catch that change (for example, the 1936 election was one of the first where class made a large difference in whether a person voted Dem of Rep).
I still trust 'random' samples more, but it's not as clear as theory would state. Therefore, we should see how the two types compare.

____________________

Geotpf:

My guess:

People who are heavy internet users tend to be richer, white, male, have higher levels of education, have a libertarian bent, etc. than the general population.

Rich, white, libertarian-leaning, highly educated males dislike Hillary Clinton more than the population as a whole. Therefore, polls heavily weighted with such people will show her doing worse than she really does. Obama does not generate a similiar reaction amoungst this demographic group, therefore his numbers are more favorable.

All self selected polls, especially self selected Internet polls, are obviously suspect.

____________________

dcshungu:

Earlier I had posted the following comment:
It is not that we are short of polling data! If anyone intends to use a flawed polling methodology for didactic purposes, s/he should present the flawed poll and discuss its flaws at the same time. As you know, polling results can drive or alter the "narrative" of a campaign. Thus, to report polling results which are known to be flawed and let spread the impression that the numbers are meaningful could alter the dynamics of an election in a way that might serve one candidate or another, but would serve neither the public nor democracy very well.M/b>

Sure enough, see how the mindless main stream media has hyped this bogus Zogby internet poll, while almost completely ignoring the more rigorous Gallup poll that paints a very different picture that is more consistent with nearly every other recent traditional poll. Do they really believe that Clinton's fortunes could have changed so dramatically so quickly?

____________________

gomer:

What about the fact the poll was taken during the Thanksgiving holiday when people are at home with their family and friends, etc? Yes, they got a high response rate but wasn't there discussion about the unreliability of polling during a holiday i.e. Iowa caucus during Christmas/New Year?

____________________

I agree with Geotpf about the internet demo being disposed against Hillary, but otherwise similar to the population at large. I think the test would be if we had more John Edwards data. I suspect the internet demo would also dislike Edwards, though not to the degree they dislike Hillary. The prevailing narrative here is that Obama wants to govern with Reason alone, rather than trying to "sell you" on an idea. My sense is that this internet demo is more than just liberterian. They might prefer minimal government, and so might tilt Republican on that score, but they might also believe that government COULD work, if only it were "done right." SO, theoretically, you have a group of people who believe that government could be robust, even expansive, provided it were carefully built and managed. It seems these people are not anti-government per se, but instead anti-rhetoric. And that's the key distinction between Obama and Clinton as far as the prevailing narrative is concerned. I would take the bottom line as "Obama appeals to engineers in a way that other Democrats don't."

____________________

Tom3:

These Zogby surveys are NOT scientific polls.

People volunteer for them. Not random calls.

Zogby is busted. They're dishonest and cannot be trusted.

____________________

Horatio Parker:

Anyone who follows the progressive media tracking sites knows that the MSM cites favorable polls even when they're wrong. Russert, for example, clung to polls that gave Giuliani an advantage over Hillary long after Hillary overtook Giuliani.

It doesn't appear that there was any evidence that this poll was designed to show HRC losing ground, but it could hardly have turned out better for those desiring that outcome. That's enough to make me suspicious.

____________________

The story here is that the Zogby numbers are being used more than others, not that Zogby's methods are necessarily bad.
As I said above, all sample results have to be adjusted because all of them are voluntary response(I actually underestimated the response rate, it's actually often below 30% for 'random' polls). What will make a sample accurate is the adjustments and that's not completely scientific, so we need to judge by the results. In the past Zogby's internet polls have been ok so they shouldn't be dismissed out of hand (remember, this isn't a poll that's just placed on a site--the people sign up on the site and an individual sample will be drawn from this group, which has hundreds of thousands of people, and then the results will be adjusted according to information given: race, sex, age, party affiliation, ....).
In this particular case, I would guess Zogby is wrong because it disagrees with most other polls.

____________________

ADG:

Charles,

Is this the first Zogby Interactive poll conducted on presidential match ups? If there are other Interactive polls asking these same questions, is there any way to highlight those data points in your graph so we could see whether the Interactive poll is typically an outlier or if this is an isolated case (e.g., I see what appear to be outliers around May/June 2007 and I'm wondering if those are Zogby Interactive or some other poll).

Thanks.

____________________

mikeel:

My apologies about my shout. What makes this poll an outlier is not so much HRC;s slippage, but how all Republicans polled are essentially statistically equal. Hillary trails them all by nearly equal margins; Obama leads them all by nearly equal margins.

Isn't that odd?

____________________

dcshungu:
mikeel wrote:

My apologies about my shout. What makes this poll an outlier is not so much HRC;s slippage, but how all Republicans polled are essentially statistically equal. Hillary trails them all by nearly equal margins; Obama leads them all by nearly equal margins.

Isn't that odd?

Astute observation that drives yet another nail into the Zogby internet-type poll's coffin. As a "pollster" John Zogby surely must be familiar with the basic requirement that the sample being polled be normally distributed to avoid biasing the results...

The striking uniformity of the responses in this poll is indeed odd, as pointed out by this poster, and is a dead giveaway that the polled sample was not "normally distributed", meaning that this was anything but a "random sampling" of opinions. The folks in this poll did not represent America. Rather, they were folks who can be distinguished by their "passionate" views and a specific ideology, and are likely to respond to an invitation to take to the internet to express those views. From these results, I would hazard the guess that the overwhelming majority of these people are far-left Democrats or far-right Republicans (the "vast left- and right-wing conspiracy") , whose only point of agreement is that they loathe Clinton with equal fervor. Presented with an opportunity to express this laothing, they repeatedly pressed the anti-Clinton "lever", and it made them feel good, at least at that moment. They were expressing their opposition to Clinton and not their support for any other specific candidate, who just as well could have been a "generic" Republican.

Let's look at the numbers again in the context of the preceding comments:

Huckabee 44%, Clinton 39% (delta: 5%)
Thompson 44%, Clinton 40% (delta: 4%)
McCain 42%, Clinton 38% (delta: 4%)
Giuliani 43%, Clinton 40% (delta: 3%)
Romney 43%, Clinton 40% (delta: 3%)


Note, as the poster pointed out, that the numbers for the Republican candidates are virtually identical and that those for Clinton are also virtually identical regardless of who she is pitted against. Also note that the delta is even largest for Huckabee who few people nationwide have ever heard of outside of the early primary and caucus states! A quick analysis of variance yields:

Mean for all Republicans 43.2 % +/- 0.8
with a variance of 0.7

Mean for Clinton: 39.4 +/- 0.9
witn a variance of 0.8

In order to get such a small standard deviation and variance from sampling something as subjective as a choice between five Republicans with "liberal" [Rudy] to conservative [Huckabee] views and one centrist Democrat, Zogby might as well have polled a bunch of clones or robots. This sample population was anything but normally distributed. It was a delta function, i.e., a "spike" within the American population continuum.

Conclusion: This poll could have pitted Clinton against anyone or anything and the results would have been the same. The deck was so stacked against her that the only possible outcome was that she'd lose.

For completeness let's look at the other match ups:

Obama 47%, Thompson 40%
Obama 45%, McCain 38%
Obama 46%, Huckabee 40%
Obama 46%, Romney 40%
Obama 46%, Giuliani 41%

Edwards 45%, Thompson 42%
Edwards 44%, Romney 42%
Edwards 44%, Giuliani 43%
Edwards 43%, Huckabee 42%
Edwards 42%, McCain 42%

Since many credible polls have shown the GOP candidates faring poorly against even a generic Democratic candidate, it would seem that the respondants of this Zogby poll might have provided a more "honest" opinion of how they feel about the other candidates. In fact, if you just assign to Clinton the GOP candidates' numbers, what you would get is a poll that qualitatively agrees with the latest and more reliable Gallup phone poll (at least for Clinton and Obama).

Dear Mr. Zogby:
Please discontinue this travesty or come up with an acceptable method for ensuring true sample randomization. As of now, all your internet poll samples is the opinion of a bunch of cyber-geeks who are "passionate" enough to respond to your invitation to take this poll. That in itself should tell you that the sample is skewed. This being a polling of opinions (a subjective quantity), no a posteriori "weighting" would effectively control for the evident a priori bias in your flawed sample "randomization" methodology.

There is no mystery here at all, and the main stream media, which must retain expensive statisticians on their staff, just got a collective black eye by hyping this bogus poll, while all but ignoring the more credible Gallup poll that showed both Clinton and Obama (but especially Clinton) doing very well against all the GOP candidates.

Howie Kurtz, here's a nice and juicy piece for your "Media Notes" column. This embarrassing episode plainly shows just how mindless the MSM can be, as already documented here

The trouble is that it is this same mindless noise machine that shapes the news that ultimately drives campaign dynamics. Way to go in service of the good ol' US Democrarcy!

DCShungu, Ph.D.
Associate Prof of Physics in Radiology,
Psychiatry, Physiology and Biophysics.
@ a couple of Ivy League Medical Schools in Manhattan.

____________________

EJP:

I smell corruption...

____________________

DTM:

dcshungu,

I would be the first to agree that the way in which many people in the press and elsewhere talk about polls is irresponsible.

But that is rather the point of pollster.com: it actually tries to talk about polls in a responsible way, and to help people become better informed about both particular polls and polling in general.

And as others pointed out, that is exactly what pollster.com did in this case, and what they have done with alternative polls in general.

Also as others have pointed out, the issues you are raising with the original sample in an internet poll are not new issues. What people have done is design various techniques to deal with these issues. Again, it remains an open question how well those techniques work, but that is something the people at pollster have been studying in depth.

So, my suggestion is that if you really want to participate meaningfully in these conversations, the first thing you should do is start reading some of the material available here about IVR and internet polls. Once you understand that background material, it will make it easier to understand pollster's discussions of specific polls, such as the one Franklin provided above.

____________________

DTM:

dcshungu,

I would be the first to agree that the way in which many people in the press and elsewhere talk about polls is irresponsible.

But that is rather the point of pollster.com: it actually tries to talk about polls in a responsible way, and to help people become better informed about both particular polls and polling in general.

And as others pointed out, that is exactly what pollster.com did in this case, and what they have done with alternative polls in general.

Also as others have pointed out, the issues you are raising with the original sample in an internet poll are not new issues. What people have done is design various techniques to deal with these issues. Again, it remains an open question how well those techniques work, but that is something the people at pollster have been studying in depth.

So, my suggestion is that if you really want to participate meaningfully in these conversations, the first thing you should do is start reading some of the material available here about IVR and internet polls. Once you understand that background material, it will make it easier to understand specific discussions, such as the one Franklin provided above.

____________________

DTM:

I apologize for the double post.

____________________

John Mitchell:

As someone who has worked in online research for the past ten years I can clarify one point about sample in this discussion - internet sample, unweighted and unbalanced, skews consistently heavily female, NOT male. Given that the result here shows a CHANGE in the results for Hillary, not a consistent trend in favor of Republicans, it is not likely to be sampling effect but some other glitch that is driving this change. Unless they goofed on their sample set up. Typically, internet sample is structured using quotas (hiss, boo, I know) in order to balance it; did they make a programming mistake this time that allowed more males in for the Hillary matchups? Who knows.

____________________

dcshungu:
One DTM wrote:

So, my suggestion is that if you really want to participate meaningfully in these conversations, the first thing you should do is start reading some of the material available here about IVR and internet polls. Once you understand that background material, it will make it easier to understand pollster's discussions of specific polls, such as the one Franklin provided above.

I am not the issue here, nor is pollster.com per se. If pollster.com wishes to "educate" the public while also doing another great public service (to help stop the spread wrong information and impression), it should post a bogus poll and simultaneously caution its readership about any methodological flaws, instead of pointing them out in a later column.

And, DTM, I have been doing "hard" scientific reseach and practicing the "scientific method" for over three decades and do not need to be lectured by you or anyone about what I should do to "understand" discussions on pollster.com. This stuff comes to me naturally, as a long-time practitioner of the "art." I hope that once for all you would just stop dripping with condescension [a well known technique to try to discredit someone] and provide your own meaningful analyses in way that would contribute to the debate. Before Franklin had come up with his analysis of this bogus poll, I had already cautioned people in a forum at Talking Points Memo blog, who were trying to hype its results, not to read anything into it. The poll was simply bogus. My "Open Letter" on this issue is part of that blog's archives.

That you are a Hillary dectractor [an established fact] and have vested interested in disparaging those who might defend her is a logical assumption. This poll was a travesty and anyone who thinks otherwise has an instant credibility problem.

Good day.

____________________

ADG asks:

"Is this the first Zogby Interactive poll conducted on presidential match ups?"

This is apparently the first Zogby Interactive poll of the presidential race this year. The Zogby release and the Reuters story compares the Internet poll with two previous Zogby phone polls from July and May.

John Mitchell-- thanks for the point on the sample bias. We don't actually know what the raw proportions of male and female were in the Zogby poll, so my comments should be viewed as a hypothetical explanation, not based on data from their poll. I should have been more clear about that.

The trick here is a purely technical explanation of what kind of sample might be expected to distort Clinton's support but not that of other dems. Since we don't know any of the sample statistics, this is pure guess work.

Charles

____________________

John Mitchell:

I just checked the Zogby sign-up. There was speculation above that bias might result from a higher level of interest in politics among a sample that chooses to respond to online political polling. The Zogby panel signup does not offer anything about the potential subject matters of the research so I am not so sure that this is the source of the bias. For all people know, they could be signing up for surveys about potato chips or washing machines.

____________________

Chris G:

I have another theory on sample bias: an online effort to drive supporters of a particular candidate or party to register for the interactive poll.

I tested this by googling "www.zogby.com/onlinepolls/"


I found 2 Ron Paul sites urging supporters to sign up to the survey:

http://dailypaul.com/node/253
http://www.ronpaulforum.com/showthread.php?t=266

Zogby doesn't report any results for Paul, and the posts are months old, but there could be a disproportionate number of Paul supporters registered for the poll. these disaffected Repubs and Inds may in turn find Clinton particularly distasteful because of her original support for the Iraq war, but more likely to support other Dems

just a theory

____________________

MarkieBee:

Could it be that Obama has better crossover appeal with Independents and Republicans than Hillary and thus, when Zogby interviews them, Obama does better against Republicans than Hillary. We all know Hillary would galvanize the Republican base against her run for the White House? With Obama as the nominee a lot of Republicans would at least give him a look. There were more than 300 Iowa registered Republicans who have pledged to caucus for Obama on Jan. 3 and more than 500 Independents and Repubs from New Hampshire have pledged the same thing. Thtis cannot be discounted and that trend likely was seen in the Zogby poll.

____________________

dcshungu:
MarkieBee wrote:

Could it be that Obama has better crossover appeal with Independents and Republicans than Hillary and thus, when Zogby interviews them, Obama does better against Republicans than Hillary.

Why then does this show up only in the Zogby internet poll and not in any anyone else's phone polls? See, e.g., the latest Gallup poll: Hillary even does slightly better Obama in identical match ups against the Republicans.

We all know Hillary would galvanize the Republican base against her run for the White House? With Obama as the nominee a lot of Republicans would at least give him a look.
This is a fantasy of non-Hillary supporters, but meaningless otherwise. There are only so many rank and file Republicans (35-40% of the electorate) and most of them would vote for their nominee and would not cross-over for Obama or any other Democrat. Since the Democratic nominee would also automatically start out with ~40% support, that leaves ~20% of so-called "swing voters" to determine the outcome of the election. These folks, who live at the center of the American political divide do not have the visceral reaction to Clinton that the far-right or far-left voters do. Therefore, there are really bot that many Republicans to galvanize in a way to would make a difference, especially if the Democrats turn out in great numbers for their candidate, and this time they will (save the maybe for the Naderite-types, who can generally be safely ignored unless the election is VERY close).
There were more than 300 Iowa registered Republicans who have pledged to caucus for Obama on Jan. 3 and more than 500 Independents and Repubs from New Hampshire have pledged the same thing. Thtis cannot be discounted and that trend likely was seen in the Zogby poll.

As you can see from the many charts on this site, the only place where Obama is competitive is in Iowa so, therefore, to cite his relative success in there(a caucus state where reliable polling is notoriously difficult to do)as somewhat indicative of his greater appeal among key swing voters is a bit of a stretch...

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR