Pollster.com

Articles and Analysis

 

Another Day, Two New Polls

Topics: 2006 , Measurement , The 2006 Race

The morning brings two new surveys: One from CBS and the New York Times (article, results, CBS analysis, results on Iraq/terror, elections) and one from Bloomberg and the LA Times (article, results). Both surveys are lengthy and the accompanying analysis of each goes into great depth on attitudes toward Congress, the upcoming elections, Iraq, and Terrorism among others. I want to take a quick look at the two measures we have been considering here for the last few days: the Bush job approval rating and the generic Congressional ballot.

First, let's add the new survey results to the table I posted last night showing how results from all national pollsters changed between August and September:

bush%20job%20mid%20sept%209-21%20update.jpg

Both surveys indicate an increase in the Bush job approval rating, although the change is bigger in the LA Times/Bloomberg poll (+5, from 40% to 45%) than in the CBS/NYT poll (+1, from 36% to 37%). Given the obvious random variation across the various pollsters in the table above, the difference is not terribly striking. While the CBS/NYT approval rating is certainly a lot lower than the LA Times rating (see Charles Franklin on "house effects"), the change since August on both polls seems within the sampling error of the average change (+2.6) seen across all eleven pollsters.

Not surprisingly, the relatively small difference in the two trends makes a big difference in the coverage. The gain in the Bush approval rating is the lead in Ron Brownstein's article:

President Bush's approval rating has reached its highest level since January, helping to boost the Republican Party's image across a range of domestic and national security issues just seven weeks before this year's midterm election, a new Times/Bloomberg poll has found.

The first reference to the Bush job rating comes in the fifth paragraph of the New York Times article by Adam Nagourney and Janet Elder and reaches a very different conclusion:

The poll also found that President Bush had not improved his own or his party's standing through his intense campaign of speeches and events surrounding the fifth anniversary of the 9/11 attacks. The speeches were at the heart of a Republican strategy to thrust national security to the forefront in the fall elections. Mr. Bush's job approval rating was 37 percent in the poll, virtually unchanged from the last Times/CBS News poll, in August.

Later the Times article offers an explanation for the contrast between this result and the apparent upswing in Bush approval reported by the recent USA Today/Gallup Poll:

The New York Times/CBS News poll began last Friday, four days after the commemoration of the fifth anniversary of the 9/11 attacks, and two weeks after the White House began its offensive on security issues. A USA Today-Gallup Poll published Tuesday reported that Mr. Bush's job approval rating had jumped to 44 percent from 39 percent. The questioning in that poll went through Sunday; The Times and CBS completed questioning Tuesday night. Presidential addresses often produce shifts in public opinion that tend to be transitory.

The obvious problem with that theory is that the LA Times/Bloomberg poll was conducted over essentially the same period (both ended on Tuesday, but the LA Times poll started a day later). A more likely explanation is simply that the real trend was slightly less than that measured by LA Times/Bloomberg and Gallup, and slightly more than that measured by CBS/NY Times. But I can also suggest one more theory. Like the Gallup poll, the CBS/New York Times poll changed the question order, adding new questions that appeared just before the job approval item at the beginning of the questionnaire:

1. I'd like you to compare the way things are going in the United States to the way they were going five years ago. Generally, would you say things are going better today, worse today, or about the same today as they were going five years ago?
     18% Better, 60% worse, 19% same, 3% DK/NA

2. And what is your best guess about the United States five years from now? Generally, if things go pretty much as you now expect, do you think things will be better, worse, or about the same as they are today?
     31% Better, 37% worse, 27% same, 5% DK/NA

These questions have appeared on CBS/New York Times surveys before, but not since January 2005. In the previous instances (January 2004 and January 2005), the questions about the direction of the country preceded the job approval question. I exchanged emails with Kathy Frankovic, director of surveys for CBS News, and she explained that "we have usually put the [state of the country questions] first (so people won't automatically connect the time frame to the President)." Most pollsters (including yours truly) use the same rationale for asking the "right direction, wrong track" question up front, just before the president's job approval rating.

Of course, it is also possible that the ordering of the most recent survey primed negative attitudes about the direction of the country and knocked down the Bush approval rating a point or two. Similar to the issue raised yesterday, without a controlled experiment that systematically compares both orderings on a survey with an enormous sample size, it would be impossible to know for certain.

Meanwhile, both surveys show Democrats holding a wide lead on the generic Congressional ballot among registered voters: The advantage is 15 points on the CBS/New York Times poll (50% to 35%), and 10 points on the LA Times/Bloomberg poll (49% to 39%). CBS also reported "similar" results among the smaller subgroup of those who say they will "definitely vote" - Democrats lead 50% to 37%. The "definite" voter subgroup is *not* equivalent to the more rigorous CBS likely voter model, which they typically use when reporting surveys conducted closer to Election Day.

Keep in mind that the result among registered voters on yesterday's USA Today/Gallup poll had the Democrats ahead by nine points (51% to 42%) among all registered voters. It was only the results obtained when applying their likely voter model that showed a significantly closer race.

 

Comments
KG:

The question that jumped out at me in the LAT poll was #49 (page 25 of the pdf) about how long the US should stay in Iraq. I don't think the results bode well for the Dems. But I haven't seen any other polls that ask similar questions, so I don't know how reliable that is.

____________________

Armando:

Could you stick to one set of respondents? It seems to me that you have included the LATimes poll's "All" instead of "RVs" numbers.

Those went from 41 to 44 on job approval.

This is not helpful to undestanding the morass of polling.

____________________

mcf:

Odd timing for the five year question to come back. The baseline skews the question.

After all, what would it take to be worse than September 2001?

____________________

Gary Kilbride:

Well, I noticed in the LA Times poll something I've been posting on many sites for months, that the economy will be a much bigger factor in November than we want to believe.

Q11. What issue will be most important to you when you decide which congressional candidate to vote for in November? Is there another issue that will be almost as important to you?

The economy leads by huge margin at 32%, with Iraq second at 21%. Interestingly, but hardly surprisingly, only among self-described liberals is Iraq the top vote-determining issue, 35% to the economy at 34%. I say not surprisingly because on liberal sites we are perpetually hammered that Iraq is the sole issue. In fact, I noticed complete silence today on those sites, regarding the finding from the LA Times on how important the economy will be in November. It's very obvious we want to bash Bush, and the Iraq war is not only a one-word convenient target, but it also makes us feel tougher, to emulate the Republican negative attacks which Democrats have been so envious of in recent cycles. It takes a hell of a lot more effort and creativity to take advantage of Bush's low approval numbers on the economy, basically identical to his approval number on Iraq in the upper 30s, so apparently we won't even try.

One poster here calls my emphasis on the economy an agenda. It's been dismissed on many sites all year. It's simply common sense. Like handicapping in 2004 that a military resume was hardly the ideal vehicle to oust a presidential incumbent, especially when that military resume was attached to a New England senator who was not particularly likable. Now we similarly want to pretend voters will largely ignore their daily lives and vote based on dusty images from far away. Iraq is built into the generic poll lead and Bush's approval numbers, and has been all year. Democrats forfeit potential advantage, which could decide the inevitable close races, by not seizing on the economic issues. I'll point out again that young single women vote in high percentages for Democrats, when they vote at all. That voting block declined for 10 years running post '92 when they decided their issues were not being addressed. That group does not make high income and economic issues are particularly significant.

____________________

Anonymous:

Well, I'm not certain this polling gets at the intensity factor (such as immigration is for some on the right) or the regional factor (immigration is more important in AZ than CT). I do know that Dems made the mistake of thinking the economy would trump national security in 2002 and 2004.

Iraq isn't the sole issue, but it may well be the most intense issue, at least regionally.

And check the CBS/NY Times numbers on the economy: 11% vs 22 on Iraq for all adults, Iraq doubles the economy amongst dems, Rs and indies.

____________________

DemFromCT:

Well, I'm not certain this polling gets at the intensity factor (such as immigration is for some on the right) or the regional factor (immigration is more important in AZ than CT). I do know that Dems made the mistake of thinking the economy would trump national security in 2002 and 2004.

Iraq isn't the sole issue, but it may well be the most intense issue, at least regionally.

And check the CBS/NY Times numbers on the economy: 11% vs 22 on Iraq for all adults, Iraq doubles the economy amongst dems, Rs and indies.

____________________

Gary Kilbride:

Sometimes I look at the polls and wonder how we're supposed to accept anything they report, when there's such blatant nonsense if you scan the details. For instance, look at some of the summary paragraphs from the LA Times poll:

* Voters living in the suburbs gave Republicans an eight point lead over their opponents last month, but in the current poll, they give the Democrats their support by 52% to 37%. Rural Americans have switched from supporting a Democratic candidate to supporting a Republican.

* Southerners moved from supporting a Republican candidate last month by 49% to 41%, to splitting their vote in the current survey.

Yeah, and within the past month I've switched from chimpanzee to octopus. Are we really supposed to believe that, the trend is toward Republicans ("in every issue, except the generic horserace question, the Republicans have improved their position") but the South is bucking it, moving from +8 GOP a month ago to dead even? Meanwhile, rural Americans have taken the reverse course, moving from Democrats to supporting Republicans. That's got to be confusing as hell if you're a rural Southerner, tugs in either direction. BTW, am I allowed to wager on which side rural Southerners will prefer?

And I'm sure suburbanites have avalanched from +8 Republican to 52-37 Democratic in one month. It reminds me of immediately post-election 2004, with exit polls spitting out brilliance like a Democrat in a 50/50 national race receiving 57% in New Hampshire. Right. And next year Tiger Woods and Roger Federer will switch sports and be equally as dominant.

That's why I appreciate the PEW surveys, with demographic changes that are subtle and sensible. Tap me on the shoulder when they report again.

And one more thing regarding the liberal sites, which I have posted on for years. The hypocrisy can be hysterical. When Bush's approval numbers were in freefall those sites were rejoicing, and embracing the lowest possible number. There was no such thing as looking at the average, or rationalizing the recent change was within the margin for error.

____________________

DemFromCT:

Gary Kilbride, I agree with you about Pew, which really to me is a gold standard poll because of all the detail provided. And I'll take the Field poll over the LA Times any day. But what I most like about pollsters is those who provide detail.. hence my fondness for this blog. You need to know the details before you do the interp.

Since I do some polling posts at a liberal blog, I disagree with your sense of "unfairness" in interpretation, but as an avowed partisan, I do enjoy Bush's slides downward, which are blessedly frequent and lasting. ;-)

But partisanship is no excuse for sloppy work, so i appreciate your comments and criticisms.

____________________

Anonymous:

"It reminds me of immediately post-election 2004, with exit polls spitting out brilliance like a Democrat in a 50/50 national race receiving 57% in New Hampshire. Right."

Those same exit polls had Kerry winning big in Connecticut, Vermont, Massachusetts, Rhode Island and Maine. And he did.

So why would you NOT believe the exit poll in New Hampshire, which borders those solid democratic states? Only if you believe there ws no fraud there.

NH has quite the history of fraud.
1) the Bush 1 surprise win in the 1988 primary over Buchanan, who had a big lead the last poll.

2) Little-known fact: The former Gov. Sununu (an electrical engineer) designed the voting machines which were used to count the votes.
Sununu had major postions in both the Reagan and Bush administrations.

3) The GOP 2004 phone campaign (spreading lies and misleading info to Dems) which resulted in several guilty pleas.

____________________

Marty H:

To 12:17PM:

"So why would you NOT believe the exit poll in New Hampshire..." you ask.

Because there was a recount. Of paper ballots. By hand. Of ~7% of the total votes. In the wards that a DU member said voting patterns indicated fraud. The recount validated the official vote tally.

That's why I believe that the exit polling in New Hampshire is wrong.

Marty H

____________________

Elizabeth Liddle:

I'd quite like to know whether the anonymous poster above thinks the exit polls in New York were correct (65.1% to Kerry).

Pre-election polls in October were: 52% (Siena); 56% (Marist); 57%, and 58% (SurveyUSA).

http://www.realclearpolitics.com/Presidential_04/ny_polls.html

Kerry won New York with 59% of the vote. All levers.

____________________

Liz,
The anonymous poster was me. I accidently submitted the post before filling out the form.

First, regarding New Hampshire. It was, as stated, a 7% partial recount of (optiscan) precincts. What about the other 93%? Shouldn't they have been recounted, as well?

Besides, 1/3 of NH precincts used punched cards. And we all know about punch cards (Florida 2000).

http://www.geocities.com/electionmodel/electionmodel_files/Election2004ExitPolls115_MachineVoteDeviations.png

Now, regarding NY.

Yes, Kerry won NY with 59%. True. All lever. True. But you imply that the 59% recorded vote is correct. In the 2004 election, that is quite an assumption to make.

Especially when:
1) It is well known that levers can be hacked.
2) You fail to account for the breakout of the late undecided vote.
3) As you have pointed out on DU, in a 155-election research study the challenger won the late undecided vote in 82% of the elections; the incumbent won in just 12%.

3) The 10/28/04 Survey USA pre-election poll had Kerry ahead by 59-37%, with 4% undecided.

4) If we assume that Kerry won 3/4 of the late undecided vote (as Harris and Zogby claim), then the adjusted poll had Kerry winning by 62/38% (4% MoE).

This is quite close to the 65% exit poll result (3% MoE).

____________________

Liz,
I just re-checked the NY pre-election and exit poll numbers and they were both off.

These are the correct numbers. I don't know where you got your 65.1% exit poll number, but it's above the 62.75% which was I have.

Pre-election:
Survey USA: Kerry 57-39-1
Projected: Kerry 59-40-1
MoE:3.9%

Exit poll:
Kerry 62.75-36.25-1
MoE:2.5%

Final (actual):
Kerry 59-40-1


1)The projection matched the actual.

2) The exit poll was off by 3.75%, which is beyond the 2.5% margin of error (3.0% assuming a 20% cluster effect).

3) With a smaller MoE, The exit poll should be considered as being closer to the true vote than the projection.

____________________

Marty H:

TIA-

You said-

"First, regarding New Hampshire. It was, as stated, a 7% partial recount of (optiscan) precincts. What about the other 93%? Shouldn't they have been recounted, as well?"

No, because the people who suspected fraud and initiated the recount process were convinced after the partial recount that the original vote tally was correct, and there was no point in continuing the obviously pointless exercise.

Look, if I told my grocer that I thumped a melon and it sounded funny, so his whole batch is bad, he'd probably ask me which melon I thumped. And if he cut it open and found a juicy, sweet, red, firm melon, I'd be inclined to say I was wrong (and pay for the melon). But if you were standing there watching us, you would want him to cut open the rest of the melons to check, and regardless of the results, you'd start complaining about the apples being wormy, and that your friend got a rotten potato from the store last year, and that the meat is rotten and that the bread is stale-all because someone else claimed that one watermelon sounded funny when it was thumped.

If there were a 100% recount in New Hampshire and it proved that the original vote tally was correct, would it change your mind about the accuracy of the exit polling? Or should your handle be "Truth is Exit Polling"?

Marty H.

____________________

Elizabeth Liddle:

Yes, I did notice that your SurveyUSA proportion was not quite right. I was going to ask where you got that. Thanks for fixing it.

In answer to your question:

The Edison-Mitofsky evaluation document

http://www.exit-poll.net/election-night/EvaluationJan192005.pdf

gives, on pages 23 and 22, the "Best Geo estimate" based on response tallies alone as 65.1%, and the "Composite Estimate" as 63.1%.

These estimates are those made at Call 3 i.e. after all tallies were in, but do not include adjustments made for the vote returns. The "Composite" estimate is a weighted average of the Best Geo and the Prior Estimate. The Prior Estimate is based on pre-election polls.

In other words, as I've said before, Edison-Mitofsky knew before a single official return was in that there was pro-Kerry bias in their exit poll because in general their Best Geo estimates for Kerry were higher than their Prior Estimates. You can see this in the Composites, as the differences between the Compositites and the count tend to be less than between the Best Geo and the count.

____________________

Elizabeth Liddle:

Marty H wrote:

"No, because the people who suspected fraud and initiated the recount process were convinced after the partial recount that the original vote tally was correct, and there was no point in continuing the obviously pointless exercise."

Also, the precincts recounted were ones that had been specifically identified as suspicious on the grounds of egregiously large swings from the previous election. So it wasn't just a case of a random melon - it was, as you say, several melons that looked a bit weird.

But the weird melons turned out OK.

As Ida Briggs wrote:

The differences between the original machine counts and the hand counts were minimal and statistically insignificant. Sometimes "odd numbers" reflect reality, and in New Hampshire, apparently that's just the way things are: larger population centers appear to be trending conservative (Republican), while the rest of the state appears to be trending liberal (Democrat).

http://www.invisibleida.com/

____________________

Mark Lindeman:

OK, TIA, so your pre-election projection has Kerry +19 in New York, the official margin is +18.3, your exit poll estimate (from screen shots?) is +26.5, and we know that the actual Best Geo estimate was +31.3.

And your projection for New Hampshire is somewhere around Kerry +2 (depending on how aggressively you allocate the undecideds), the official is Kerry +1.4, you probably have the exit poll at about Kerry +11, and we know that the actual Best Geo estimate was +15.0.

Is that about right?

Hmmmm. Yeah, who can fathom those crazy naysayers who doubt the exit polls?

____________________

Mark,

I'll stick by my numbers; you can keep your Geo estimates. You try to make it appear that the NYS exit poll was so far off the pre-election poll.

Well, the pre-election poll MoE was 3.89% with the same deviation from the exit poll and the vote: 3.75%.

Exit Poll: 62.75
Kerry Actual: 59

The NYS exit poll MoE was 2.47% assuming zero cluster; 2.96% assuming 20%; 3.21% assuming 30%.

I trust the exit poll more than the pre-election poll. So Kerry true vote was probably
around 62%. Gore won NY by 60-35%, with 5% to Nader.

NY was one of 16 states in which the exit poll was exceeded by the vote count beyond the
margin of error for Bush. None to Kerry.
The Probability: 1 in 19 trillion.

____________________

Mark Lindeman:

Good golly, sir, it's not a matter of what I'm "try[ing] to make it appear." The New York exit poll actual was 65.1, not 62.75. As for the pre-election polls, there were four of them in the last week, all within a few points of each other on the margin. Needless to say, none was anywhere near the +31.3 in the exit poll.

If you stop selectively hiding behind screen shots and margins of error, the problem is pretty clear. The official results in both New York and New Hampshire are quite close to the pre-election polls, and the exit polls are way out. But of course, if you believe in screen shots and screen shots are all you believe, then there is really nothing to talk about.

Gary, people may set out to vote the economy in November, but I'm not sure what they will actually be voting. It should be interesting to watch.

____________________

The Dems have held a steady 10 point lead in the Generic Vote Polls over the past year.

The average poll had a 3.5% MoE. So, if you believe the MoE's published by the pollsters, the Dems lead cannot be just a series of random occurrences. So their chances of winning the House and/or Senate are excellent.

In fact, if the election were held today, there are only two ways they could lose.

1) Divine intervention
2) Massive fraud

Which is more likely to occur? I believe it will be number 2. But this time, Rove's job will be harder.

So why the hell do we even bother to track the polls?

Here are 86 generic polls from polling report.com

http://www.geocities.com/electionmodel/GenericPollTrend_15084_image001.png

Survey Dates REP DEM Other Diff Rep10MA Dem10MA Diff
.
CBS/New York Times RV 9/15-19/06 35 50 15 15 39.6 49.4 9.8
USA Today/Gallup LV 9/15-17/06 48 48 4 0 39.3 49.1 9.8
FOX/Opinion Dynamics LV 9/12-13/06 38 41 21 3 38.8 49.5 10.7
Gallup RV 9/7-10/06 41 53 7 12 39.5 50.1 10.6
Pew RV 9/6-10/06 39 50 11 11 38.7 48.8 10.1

ABC RV 9/5-7/06 42 50 9 8 38.9 48.8 9.9
CNN LV 8/30 - 9/2/06 43 53 4 10 38.6 48.9 10.3
FOX/Opinion Dynamics RV 8/29-30/06 32 48 21 16 38.4 48.6 10.2
Newsweek RV 8/24-25/06 38 50 12 12 38.2 48.6 10.4
Time AV 8/22-24/06 40 51 9 11 38.1 49.1 11.0

CBS/New York Times RV 8/17-21/06 32 47 21 15 38.0 49.2 11.2
CNN RV 8/18-20/06 43 52 6 9 38.8 49.8 11.0
USA Today/Gallup RV 8/18-20/06 45 47 7 2 38.5 49.7 11.2
Diageo/Hotline RV 8/17-20/06 33 40 27 7 37.5 49.5 12.0
Pew RV 8/9-13/06 41 50 9 9 37.4 50.3 12.9

Newsweek RV 8/10-11/06 39 51 10 12 36.7 49.5 12.8
USA Today/Gallup RV 8/7-10/06 41 50 9 9 36.8 49.5 12.7
FOX/Opinion Dynamics RV 8/8-9/06 30 48 22 18 36.8 49.6 12.8
AP-Ipsos RV 8/7-9/06 37 55 8 18 37.3 49.5 12.2
ABC/Washington Post RV 8/3-6/06 39 52 8 13 37.4 49.4 12.0

CNN RV 8/2-3/06 40 53 7 13 37.4 49.4 12.0
USA Today/Gallup RV 7/28-30/06 40 51 8 11 37.0 48.2 11.2
CBS/New York Times RV 7/21-25/06 35 45 20 10 36.9 48.2 11.3
Diageo/Hotline RV 7/20-23/06 32 48 20 16 37.2 48.2 11.0
FOX/Opinion Dynamics RV 7/11-12/06 34 42 25 8 37.8 48.2 10.4

AP-Ipsos RV 7/10-12/06 40 51 9 11 37.7 48.7 11.0
Gallup RV 7/6-9/06 41 51 9 10 37.6 48.7 11.1
Time AV 6/27-29/06 35 47 18 12 37.7 48.7 11.0
USA Today/Gallup RV 6/23-25/06 38 54 7 16 37.8 48.1 10.3
ABC/Washington Post RV 6/22-25/06 39 52 9 13 37.6 46.4 8.9

Diageo/Hotline RV 6/21-25/06 36 41 24 5 37.7 46.4 8.8
Pew RV 6/14-19/06 39 51 10 12 38.0 47.4 9.4
CNN RV 6/14-15/06 38 45 16 7 37.3 46.7 9.3
FOX/Opinion Dynamics RV 6/13-14/06 33 46 20 13 37.4 46.8 9.4
USA Today/Gallup RV 6/9-11/06 39 51 10 12 38.0 47.6 9.6

USA Today/Gallup RV 6/1-4/06 42 51 7 9 37.3 46.9 9.6
Diageo/Hotline RV 5/18-21/06 36 42 22 6 37.1 46.8 9.7
Fabrizio, McLaughlin (R) LV 5/15-17/06 36 39 25 3 37.6 47.7 10.1
ABC/Washington Post RV 5/11-15/06 40 52 9 12 38.2 49.0 10.8
Newsweek RV 5/11-12/06 39 50 11 11 38.2 49.3 11.1

CBS/New York Times RV 5/4-8/06 33 44 23 11 37.7 48.7 11.0
CNN RV 5/5-7/06 38 52 10 14 38.5 49.3 10.8
FOX/Opinion Dynamics RV 5/2-3/06 38 41 21 3 38.6 49.1 10.5
USA Today/Gallup RV 4/28-30/06 39 54 7 15 38.5 50.2 11.7
Cook/RT Strategies 4/27-30/06 32 44 24 12 38.5 50.3 11.8

CNN RV 4/21-23/06 40 50 9 10 38.7 50.7 12.0
Pew RV 4/7-16/06 41 51 8 10 38.6 51.0 12.4
USA Today/Gallup RV 4/7-9/06 42 52 6 10 38.5 50.7 12.2
ABC/Washington Post RV 4/6-9/06 40 55 5 15 37.4 50.1 12.7
CBS RV 4/6-9/06 34 44 22 10 37.5 49.2 11.7

Time RV 3/22-23/06 41 50 9 9 38.4 49.8 11.4
Newsweek RV 3/16-17/06 39 50 11 11 38.4 49.8 11.4
NPR LV 3/12-14/06 37 52 11 15 38.3 50.2 11.9
CNN/USA Today/Gallup RV 3/10-12/06 39 55 7 16 38.7 49.9 11.2
FOX/Opinion Dynamics RV 2/28 - 3/1/06 34 48 18 14 38.2 48.7 10.5

CNN/USA Today/Gallup RV 2/28 - 3/1/06 39 53 7 14 39.1 48.8 9.7
Democracy Corps (D) LV 2/23-27/06 40 48 12 8 39.3 48.6 9.3
Diageo/Hotline RV 2/16-19/06 31 46 23 15 39.0 48.3 9.3
GWU Battleground LV 2/12-15/06 41 46 14 5 39.2 48.0 8.8
CNN/USA Today/Gallup RV 2/9-12/06 43 50 8 7 39.2 48.3 9.1

Pew RV 2/1-5/06 41 50 9 9 38.2 47.5 9.3
ABC/Washington Post RV 1/23-26/06 38 54 9 16 38.0 47.4 9.4
Democracy Corps (D) LV 1/22-25/06 41 49 10 8 37.8 46.8 9.0
CBS/New York Times RV 1/20-25/06 34 43 23 9 37.8 46.7 8.9
CNN/USA Today/Gallup RV 1/6-8/06 43 49 8 6 37.9 46.5 8.6

ABC/Washington Post RV 12/15-18/05 41 51 9 10 37.2 46.9 9.7
NPR LV 12/15, 17-18/05 37 45 17 8 37.1 46.6 9.5
Diageo/Hotline RV 12/12-13/05 33 43 25 10 37.1 47.3 10.2
Democracy Corps (D) LV 12/8-12/05 41 49 9 8 38.1 48.0 9.9
CBS/New York Times RV 12/2-6/05 33 42 25 9 37.9 47.9 10.0

Democracy Corps (D) LV 11/30 - 12/4/05 39 49 12 10 38.4 48.6 10.1
Time RV 11/29 - 12/1/05 36 48 15 12 37.6 47.6 10.0
Democracy Corps (D) LV 11/16-20/05 41 48 11 7 38.1 47.4 9.3
Diageo/Hotline RV 11/11-15/05 35 41 24 6 38.1 47.2 9.1
Newsweek RV 11/10-11/05 36 53 11 17 38.9 47.9 9.0

Democracy Corps (D) LV 11/2-6/05 40 48 12 8 39.2 47.3 8.1
ABC/Washington Post RV 10/30 - 11/2/05 37 52 12 15 39.2 47.8 8.6
CNN/USA Today/Gallup RV 10/21-23/05 43 50 7 7 39.3 47.6 8.2
Democracy Corps (D) LV 10/19-23/05 39 48 12 9 38.9 47.3 8.4
Diageo/Hotline RV 10/12-16/05 31 40 29 9 38.9 47.1 8.3

GWU Battleground LV 10/9-12/05 41 47 13 6 40.2 48.3 8.2
Democracy Corps (D) LV 10/5-10/05 41 46 14 5 40.0 48.6 8.6
Newsweek RV 9/29-30/05 42 47 11 5 39.8 49.3 9.5
Democracy Corps (D) LV 9/19-21/05 39 48 13 9 39.0 50.0 11.0
Pew RV 9/8-11/05 40 52 8 12 39.0 51.0 12.0

Newsweek RV 9/8-9/05 38 50 12 12 38.0 50.0 12.0

____________________

Elizabeth Liddle:

TIA: You may turn your nose up at the "Best Geo" estimates in the E-M report all you want, but the fact remains that they are the actual estimates made on the basis of the exit poll responses alone, and they show a massively significant discrepancy between the exit poll and the count. The "Composite" estimates, which are closer to your screen shots (because the screenshots would have been largely based on them, plus, possibly, a smidgeon of vote-returns), are also closer to the count, because they were weighted by pre-election polls.

So it is, frankly, ludicrous to insist that your screenshot exit poll numbers are both more accurate reflections of the exit poll numbers and closer to the pre-election polls than the Best Geo, because the reason they are closer is that they reflect Composite estimates that were actually adjusted towards pre-election polls, on the assumption that there must have been pro-Kerry bias in the exit poll. And yet, elswhere, you insist that such bias is impossible, and that any non-response bias in the exit poll could only have been pro-Bush.

So you can't have your cake and eat it. Either you accept the screenshot/Composite estimates instead of the Best Geo, and, with that acceptance, also accept, as the pollsters did, that there was pro-Kerry bias in the exit poll responses, or you go for the "pure" Best Geo, and accept that the exit polls for New Hampshire and New York (and a few others) were even more way out of whack with both pre-election polls and count.

Polls, as you yourself point out when it suits your case, can be biased. This can happen because of the way questions are framed, or samples designed, and can reflect the political assumptions of the pollster or their client. It can also happen for a host of other reasons, including non-response and selection bias. Selection bias is a particular hazard when respondents are selected by means of a face-to-face approach, as in the exit polls. Abundant evidence, including the disparity between the Best Geo and your own New York data, suggests that the exit poll had a pro-Kerry bias (though it cannot tell you how large).

To insist, as you appear to, that only pro-Republican bias is possible is not only a biased view in itself, but flies in the face of copious actual evidence. And to insist that when pollsters give an MoE to reflect sampling error that that implies they believe that sampling error is the only possible source of error in their poll is simply wrong. Pollsters are as aware of potential sources of non-sampling error in polls as a doctor is aware (or should be!) of the wide range of unpredictable factors that can affect their prognoses.

To quote Warren Mitofsky himself:


    I want to say a few words about reporting sampling error. A number of people who have spoken here have talked of not reporting sampling error because it was confusing all those dear mindless souls who listen to our results. They were concerned we would make people think that sampling error was the only error in the survey. I guess I am not too sympathetic with that point of view.

http://www.nyaapor.org/WMitofskySpeech.htm. (my bold)

If you actually want to learn about polling (as opposed to propound analyses that assume that sampling error is the only error in polls) then I suggest you actually read some of the posts on this site. Polling is a fascinating subject, as is the information it gives us, but there is a heck of a lot more to it than the binomial theorem.

Cheers

Lizzie.

____________________

Liz,

There you go again: claiming the polls were biased for Kerry and that I maintain they are perfect random samples. To both I say: hogwash.

Exit poll bias is apparently a one-way street to you; it always favors the democrat. Well, you are partially correct: 75% of the spoiled ballots which are never counted (around 2-3% of the votes) are democratic.

Now, let's look at some other biases.

1) The Income factor

Low-income Kerry non-responders outnumbered high-income Bush non-responders.

[link]

2. State exit poll completion rates

Completion rates decline going from strong Bush Bush to strong Kerry states.

[link]

3) The Final National Exit Poll

According to the poll, which was matched to the vote count, 43% of the 122.3mm who voted in 2004 were Bush 2000 voters and just 37% were Gore voters. So we can conclude that non-responders were Kerry voters. What does that tell us about the rBr theory? Mitofsky refutes it himself in the final exit poll.

But...the 43% Bush weighting was mathematically IMPOSSIBLE. It implies that he had 52.57mm votes in 2000; he had just 50.45mm - and about 1.75mm of them died before the 2004 election. So at MOST 48.7mm Bush 2000 voters (or 39.8% of 122.3mm) could have turned out to vote in 2004.

Therefore, the Final NEP was impossible. FOUR million phantom votes were added to the Bush total and subtracted from Kerry's. It's a slam dunk: Since the Final NEP results were impossible and since the final was matched to the vote, so, too, the recorded vote was impossible...

Q.E.D.

But a NEW explanation to overcome the impossible final NEP math is to jettison the refuted rBr hypothesis and claim that the 43% result was due to "false recall" on the part of Gore voters who said they voted for Bush. There must have been 6% more (7.5mm) Gore voters who forgot that they voted for Gore than Bush voters who forgot they voted for Bush. That is only arrow left in the quiver to explain why the exit poll discrepancy does not prove that the election was stolen. We must believe the "false recall" argument: that 7.5 million more Gore voters than Bush voters forgot who they voted for in 2000 when they came to the polls in 2004.

Well, not quite. You now say that the exit polls were faulty, citing distance from the polling booth, inexperienced, young surveyors (weren't most of the pollsters over 60 Bush voters?).

Ok, then, what about the MoE provided for each generic and pre-election poll - is it irrelevant, since the polls are not PERFECT random samples? If your answer is yes, then why should we believe ANY poll? And why quote ANY MoE?

Projecting the Kerry vote based on 18 national pre-election polls and the 50 state polls, the MoE for each of the COMBINED state and national pre-election poll groups was under 0.6%.

The 18 final national polls had slightly different results and had 2.5-3% MoEs. They were a mix off 9 RV and 9 LV polls. Kerry led in 11 of them.

NATIONAL PRE-ELECTION 18-POLL SUMMARY:
Kerry won 11, Bush 6, 1 tie
Kerry won 5 of 9 Registered Voter (RV) Polls
and 6 of 9 Likely Voter (LV) Polls

Total Poll Total Weighted Average
Sample Sample MoE KERRY BUSH
Date 26961 Group 0.60% 47.80 47.14

1-Nov Marist 1166 LV 2.87% 50 49
1-Nov Econ 2903 RV 1.82% 50 47
1-Nov TIPP 1284 LV 2.73% 44 47
1-Nov CBS 1125 RV 2.92% 47 48
2-Nov Harris 1509 LV 2.52% 50 47

2-Nov Zogby 1200 LV 2.83% 50 47
31-Oct FOX 1400 RV 2.62% 48 45
31-Oct DemC 1018 LV 3.07% 48 47
31-Oct Gallup1866 RV 2.27% 48 46
31-Oct NBC 1014 LV 3.08% 47 48

31-Oct ABC 3511 RV 1.65% 47 48
30-Oct ARG 1258 LV 2.76% 49 48
30-Oct Pew 2408 RV 2.00% 46 45
29-Oct Nwk 1005 RV 3.09% 44 48
26-Oct ICR 817 RV 3.43% 48 48

24-Oct LAT 1698 RV 2.38% 48 47
21-Oct Time 803 LV 3.46% 46 51
20-Oct AP 976 LV 3.14% 49 46

There were 51 state pre-election polls, each with a 4% MoE. The combined weighted MoE was under 0.4%. Kerry led Bush here as well.

[link]

In the pre-elects, there was no cluster effect, no question as to the interviewer being young, seeking out democrats, no false recall. So where is the bias?

So it comes down to this. Either
a) "false recall" explains the discrepancies or
b) the pre-election and 12:22am exit polls were essentially correct: Kerry won with 51-52% of the vote.

____________________

Mark Blumenthal:

For now, this site is set up to allow for the use of limited HTML tags (bold, italics, hyperlinks) in comments and to automatically create URL links from "bare" URL's copied into comments.

However, Movable Type, the software on which this blog runs, also uses an automated system to block comment SPAM. The system will automatically hold certain comments for approval when they exceed a "junk threshold." It appears that pasting a large number of bare URL's into a comment leads to a high "junk" score and forces such approval of comments before they appear here.

As such, we urge commenters to be sparing in the use of pasted URLs.

Please note that I also edited the previous comment to transform several long bare, autolinked URLs to a single linked word ("[link]") so that they do not run into the next column.

____________________

Elizabeth Liddle:

Well, if you do not believe they are perfect random samples, why are you so reluctant to conclude that polls outside their own MoE might be biased?

I certainly don't maintain that bias always favours the Democrats. There are polls that appear to favour Republicans (as you yourself have noted). Indeed there were exit polls that favoured Republicans. I have no preconceived idea whatsoever that bias must go one way rather than the other. To find out whether a poll was biased one way rather than another, or not at all, you have to look at the data.

And the exit poll data indicates that the "Composite" estimates (on which your screenshots were based), which were weighted by pre-election polls, tended to produce a lower value for Kerry than the Best Geo value (based on the exit poll responses). From this we learn that the E-M Prior Estimates of Kerry's voteshare (based on pre-election polls) must have tended to be lower than their exit poll responses.

Now, it could have been that their pre-election estimates for Kerry were too low; or it could have been that their exit poll responses were too high. But at least one of the two was consistently biased. However, for New York, at least, we can infer that their pre-election poll was in the same kind of ball-park as yours and the other pre-election polls, and it was also close to the count. Therefore, the most parsimonious explanation is that it was the exit poll that was off. In which case the bias was pro-Democrat. Ditto with NH.

Re Gore voters I shall simply refer you yet again to:

http://inside.bard.edu/~lindeman/too-many.pdf

But I will address your next question:

Why should we believe any poll or quote any MoE?

It's a very good question. But in partial answer, I suggest you read Mitofsky's address (see link above) for a good rationale for providing an MoE. I certainly wouldn't want to start to interpret any poll without that information as it is important. It tells us what variance to expect from sampling error alone, and as you frequently and rightly point out, the larger the sample the less this component is likely to be. If a pollster didn't provide it, I would calculate it myself from the sample size. And if they didn't provide a sample size I would ignore the poll.

However, to evaluate a poll you also need to evaluate likely non-sampling sources of error, particularly of bias. It's one of the reason I come so often to this excellent site.

There are several approaches you can take: one is to track specific pollsters for trends. Even if a particular pollsters's methodology means that their results track too high (or low) for your liking, within-pollster upturns or downturns may nonetheless reflect real changes, especially if reflected in other lower(or higher) tracking polls. And if not, it is worth seeing if there has been any change in methodology (for example, a change in wording or question order) that might account for the blip.

Another approach is to look carefully at the methodology itself, and evaluate that (as indeed you do when you consider RV versus LV models, and assumptions regarding undecideds). And of course that is exactly what E-M did (and I did with their data) when evaluating the 2004 exit poll discrepancy. You can, if you like, regard each precinct as a mini poll, each with its own methodology, which varied (for example, sampling rate). If certain methodological factors were associated with larger poll-count discrepancies (and they were), you can conclude that that factor was likely to have been responsible for the higher Kerry (for instance) proportion in the poll at that precinct.

Regarding "false recall" and the discrepancy: certainly misreporting of 2000 vote doesn't explain the discrepancy. It just looks like a small but significant phenomenon affecting the "past vote" question only. What fits the data is that there was pro-Kerry bias in the poll arising from an underlying differential propensity for each group of voters to participate, and that this was more likely to be translated into bias if conditions at the poll were such as to make non-random selection more likely.

Finally, you ask for an explanation for bias in the pre-election polls. I dispute that there was overall bias. Kerry led in your own model, but your own model made fairly extreme assumptions regarding allocation of undecideds - clearly any meta-analysis is going to depend on the accuracy of the assumptions as well as on the size of the samples. As Sam Wang said - you need to put a probability on your own assumptions being correct, although the MoE on those will depend entirely on your preparedness to consider being in error.

The pollsters' own assumptions look somewhat more in line with the results:

http://www.realclearpolitics.com/bush_vs_kerry.html

____________________

Liz,

It's interesting that you provide link to realclear.com. It's always been real clear to me that it is a Republican-leaning site. Just take a look at the polls listed- all LVs. But you simply ignored that fact.

"Kerry led in your own model, but your own model made fairly extreme assumptions regarding allocation of undecideds."

Extreme undecided voter assumptions?
I had it at 60-75%%. Harris at 2-1 to 4-1.

The 12:22am exit poll had it at 5-4 to 3-2:

WHEN DECIDED
Votes Pct Kerry Bush Other Total
Today 7.31 6% 53% 40% 5% 98%
3Days 3.65 3% 53% 41% 4% 98%
7days 2.44 2% 48% 51% 1% 100%
30days 12.18 10% 60% 38% 1% 99%
Over30 96.24 79% 50% 50% 0% 100%

And here is the result of a 200 trial pre-election and exit poll simulation of both state and national polls, in which I assume 67% of the undecided vote went to Kerry.

http://www.geocities.com/electionmodel/InteractiveElectionSimulation_12255_image001.png

As I have stated time and again, in 2004 RV polls were more accurate than LVs, simply because there was massive registration of new Democratic voters. And they were by definition RVs, not LVs.

In my Election Model, I used a mix of 9 RVs and 9 LVs. And again, it's real clear why you chose to cite Realclear.com. You may have seen it referenced by others who also tried, unsuccessfully, to debunk my pre-election analysis, saying I cherry-picked the polls.

Such hypocrisy: There is not ONE RV poll quoted in that Real Clear list. And they say I cherry-picked. The realclear.com site is real clear evidence of pre-election poll cherry-picking.

Now, as far as LV vs. RV polls, check out Pew's final pre-election polls:
http://people-press.org/reports/images/232-1.gif

By the way, the Election Model projections (both state and national), along with Sam Wang's, were the only ones which matched the exit polls to within 0.5% - and both projected a solid Kerry EV win with a 99% probability.

As far as the "professional" political science forecasts were concerned, let's look at the record. All the models presented here were not even close to the recorded vote, much less the TRUE vote.

http://www.apsanet.org/content_13000.cfm

Except for Beck/Tien. Even if they were off thefrom the exit polls by 1.50% when they called it a dead-heat, their 50% Kerry win probability made sense based on the prediction.

All others were way off, not only in their Bush vote predictions, but in the associated probabilities. Just run the numbers.

A 53% 2-party vote projection equates to a 97.5% win probability, assuming a 3.0% MoE. At 54%, a 3.0% MoE implies a 99.9% probability.

Apparently, the forecasters did not have too much confidence in their projections.

Author Pick 2-pty Date Win Probability
Abramowitz Bush 53.7% 7/31/04 -
Campbell Bush 53.8% 9/06/04 97%
Wlezien/Ericson Bush 52.9% 7/27/04 75%
Holbrook Bush 54.5% 8/30/04 92%
Beck/Tien Kerry 50.1% 8/27/04 50%
Lockabie Bush 57.6% 5/21/04 92%
Norpoth Bush 54.7% 1/29/04 95%

Check the final prediction dates. The latest was 9/6/04, a full TWO months BEFORE the election.

Their combined Bush 2-party average 54% projection isn't even close to the recorded vote, much less the true vote. It's off by more than 5%, if, as the exit polls say, Bush got 48.5% of the two-party vote. Well, that's if you believe my election model. And the exit polls.

There's no way Lockabie's 57.6% Bush popular vote equates to anything less than a 100% EV and/or popular vote win probability. His 92% Bush win probability doesn't match his projected vote share. At a one-tail (97.5%) level of confidence, it implies an equivalent MoE of +/-7.6% based on the 57.6% projection. Not good, especially when winning 52% of the popular vote means there is a virtual 100% probability of winning the electoral vote. The same goes for the rest of the models.

The forecasters should have known that things change every day in a presidential horse race. But they never updated their projections. As we all know, or should know by now, the electorate is a dynamic organism. Things change every day.

Couldn't the poli-scientists have run their models right up to Nov.1, 2004? I did. I ran the Election Model almost every day during the four months leading up to the election. And here is the proof:

http://www.geocities.com/electionmodel

Just click on the graphs to see the projected trend in vote share, electoral vote and win probability. These statistics changed slightly each time I updated the database for the latest poll results.

I used pollingreport.com for the national polls and electoralvote.com for the states.

____________________

Elizabeth Liddle:

Well, I certainly deny the charge of hypocrisy. Desist, please.

My point was simply that the answer you get depends on assumptions, not on MoEs.

The LV assumption might have been wrong, but it was in line with results. Your assumptions might have been wrong, and although overall your vote proportions were approximately equal to the overall unweighted exit poll response proportions, they don't come near to matching state-by-state. The exit poll deviations tend to be greater and in different states, regardless of whether you use RV or LV models.

As in both New York and New Hampshire.

____________________

Liz,

Cool it. The hypocrisy was that of those who said that I cherry-picked. It was not aimed at you. I don't recall you ever saying it.

"The LV assumption might have been wrong, but it was in line with results. Your assumptions might have been wrong, and although overall your vote proportions were approximately equal to the overall unweighted exit poll response proportions, they don't come near to matching state-by-state. The exit poll deviations tend to be greater and in different states, regardless of whether you use RV or LV models".

That is a false statement. The pre-election state polls DID match the exit polls. Why won't you just accept it? I show all my numbers. You just make a flat statement, without citing the numbers.

Here's proof they matched.

Base case assumption:
60% of Undecided Vote allocated to Kerry
Calculate:
Kerry's pre-election state poll projections and compare then to the exits
(ignore third party vote shares).

The key results:

Sensitivity of net average deviation to pre-election Kerry undecided allocation.

Undec % 50 55 60 67 75
NetDev% 0.26 -0.05 -0.37 -0.81 -1.31

1. There was a 0.37% deviation (assuming 60% undecided to Kerry) between the average state pre-election poll and the average exit poll.

2. There was a 0.05% deviation (assuming 55% undecided to Kerry) between the average state pre-election poll and the average exit poll.

3. In 39 states, one individual led in both the pre-election and exit poll.

4. In 15 of 17 battleground states, one individual led in both the state and exit poll.

5. In 15 states, pre-election vote shares differed from the corresponding exit poll vote shares by less than 1%;
in 29 states, by less than 2%;
in 32 states, by less than 3%;
in 42 states, by less than 4%.

And you say they don't match? What do you want, absolute agreement between the pre-election and exit poll for each state? To any impartial observer, there is a very close match, indeed.

6. When weighted by voting population,the pre-election projection is:
Kerry-50.37%, Bush- 48.63%, Other-1%.

7. Kerry's 2-party exit poll is 50.51%.

Liz, a question: Since you do not believe the exit polls and believe there was a net Kerry bias, does the close match shown here between the state pre-election and exit polls imply that the pre-election polls had to be biased for Kerry as well?

Table Column headings:
KPoll: Kerry pre-election poll
Kproj: Kerry pre-elect + undecided allocation
Kexit: Kerry Exit poll
Kact: Kerry Actual vote

KpDev: Kproj-Kact
KeDev: Kexit-Kact
NetDev:KeDev-KpDEv

Kerry's UNWEIGHTED projections and poll numbers are shown in the following table.

[code]
. Kpoll Kproj Kexit Kact KpDev KeDev NetDev
Avg 48.5 49.2 48.8 47.1 2.12 1.75 -0.37
Median 49.5 50.0 49.1 47.5 2.01 1.83 -0.32

AL 40.6 41.4 41.1 37.1 4.30 3.98 -0.32
AK 34.5 37.8 40.1 36.8 1.03 3.37 2.34
AR 47.4 48.0 46.6 45.1 2.93 1.53 -1.40
AZ 48.9 49.6 46.9 44.7 4.88 2.21 -2.67
CA 53.8 54.4 55.7 55.0 -0.64 0.69 1.33

CO 49.5 50.0 49.1 47.6 2.37 1.44 -0.93
CT 55.3 55.6 58.5 55.3 0.33 3.20 2.87
DE 54.2 55.2 58.4 53.8 1.37 4.61 3.24
DC 87.6 84.6 91.6 90.5 -5.92 1.11 7.03
FL 50.0 50.6 49.9 47.5 3.12 2.45 -0.67

GA 44.7 45.6 43.1 41.6 3.95 1.46 -2.49
HI 50.0 51.0 53.3 54.4 -3.40 -1.08 2.32
ID 33.7 36.6 33.3 30.7 5.92 2.66 -3.27
IL 56.3 56.4 57.1 55.2 1.19 1.93 0.73
IN 40.2 40.8 41.0 39.6 1.22 1.39 0.17

IA 53.2 53.6 50.7 49.7 3.94 1.01 -2.93
KS 38.1 38.8 34.6 37.1 1.67 -2.53 -4.20
KY 41.1 42.0 40.8 40.0 2.01 0.76 -1.24
LA 45.5 47.2 44.5 42.7 4.53 1.83 -2.70
ME 56.2 56.6 54.8 54.6 2.02 0.25 -1.77

MD 55.7 55.8 57.0 56.6 -0.77 0.47 1.24
MA 70.3 69.4 66.5 62.7 6.66 3.72 -2.94
MI 53.6 53.8 52.6 51.7 2.07 0.83 -1.25
MN 54.2 54.4 54.6 51.8 2.64 2.85 0.21
MS 45.2 46.2 43.2 40.5 5.71 2.71 -3.00

MO 47.3 48.2 47.5 46.4 1.82 1.09 -0.73
MT 38.7 40.2 39.3 39.5 0.70 -0.22 -0.92
NE 34.4 36.2 36.5 33.2 3.05 3.39 0.34
NV 50.0 50.2 50.7 48.7 1.52 1.98 0.46
NH 50.0 50.6 55.5 50.7 -0.09 4.80 4.89

NJ 54.3 54.8 56.1 53.4 1.43 2.76 1.33
NM 50.0 50.2 51.3 49.6 0.60 1.74 1.14
NY 59.4 59.4 64.0 59.3 0.11 4.68 4.57
NC 48.5 48.8 47.3 43.8 5.04 3.55 -1.49
ND 38.9 41.0 33.6 36.1 4.91 -2.51 -7.42

OH 51.5 51.8 52.1 48.9 2.86 3.12 0.26
OK 31.5 34.6 34.7 34.4 0.17 0.30 0.13
OR 53.2 53.6 51.2 52.1 1.49 -0.89 -2.38
PA 52.6 53.0 54.4 51.3 1.74 3.15 1.41
RI 60.9 60.8 64.2 60.6 0.22 3.66 3.44

SC 43.3 43.8 45.8 41.4 2.44 4.42 1.98
SD 44.7 45.6 37.4 39.1 6.51 -1.67 -8.18
TN 48.5 48.8 41.2 42.8 5.99 -1.66 -7.65
TX 38.5 39.4 36.8 38.5 0.91 -1.65 -2.56
UT 25.8 28.2 29.9 26.7 1.55 3.28 1.73

VT 57.0 57.2 65.7 60.3 -3.10 5.39 8.49
VA 48.0 48.2 48.0 45.9 2.33 2.09 -0.24
WA 54.2 54.4 55.1 53.6 0.75 1.42 0.67
WV 47.9 48.6 45.2 43.5 5.08 1.67 -3.41
WI 53.7 54.0 50.2 50.2 3.81 0.02 -3.79
WY 30.9 32.6 32.1 29.7 2.91 2.38 -0.53

[/code]
______________________________________________

____________________

Mark Lindeman:

TIA, I assume you know very well that the "Republican-leaning site" bit here is (generously) a weak distraction. Elizabeth could just as well have linked to pollingreport.com, with substantively the same result.

What's "real clear" is that you aren't embarrassed to cite a Pew study headlined "Slight Bush Margin in Final Days of Campaign" as evidence that Kerry was ahead. Nor are you embarrassed to assert without an iota of support that new Democratic registrants were "by definition... not LVs." Similarly, you aren't embarrassed to ask a rhetorical question about election forecasting models that betrays ignorance or indifference as to their substance. What is less clear is why you aren't embarrassed.

____________________

Elizabeth Liddle:

TIA, I cited my numbers - the Best Geo estimates in the E-M report. Those are the numbers from the "pure" exit polls, if you like, unweighted by pre-election polls. Sure, if you use yours, and also use your pre-election projections, the two sets of deviations (eyeballing your chart) look to have both similar variance and similar means.

But eyeballing your own chart, you can see that there is simply no statewise match. Look at Vermont, DC, New Hampshire, New York - exit polls way ahead of the pre-election polls. Then look at North and South Dakota, and Tennessee, all with substantial blueshift relative to the exit poll but a count that was 5 or 6 points "redder" than the pre-election poll. And then, having eyeballed them, actually do the correlation. There is no correlation between the two sets of deviations.

No, I don't expect "absolute agreement" between the two sets of deviations, but if you are going to make the case that both sets had the same cause - fraud in the count - then a sniff of a positive correlation between the two would be helpful. But there isn't one.

Matching the means tells you very little, especially when the mean of the pre-election polls are subject to debate regarding assumptions, and you are using exit poll data that is already weighted to pre-election polls.

But I appreciate your reassurance that I was not the target of your "hypocrisy remark".

Cheers

Lizzie

____________________

Elizabeth Liddle:

Sorry, missed your question:

"Liz, a question: Since you do not believe the exit polls and believe there was a net Kerry bias, does the close match shown here between the state pre-election and exit polls imply that the pre-election polls had to be biased for Kerry as well?"

I don't "believe" anything. I weigh up data. I consider that the data strongly support the hypothesis that the exit polls, overall, had a pro-Kerry bias. As far as the pre-election polls are concerned - I don't know whether the polls themselves were biased. Some may have been. However, as you know, the problem with pre-election polls is that as well as the possibility of bias in the poll itself (e.g. non-response bias; selection bias), the estimate will depend on assumptions made regarding which of your respondents will actually vote, and who those who say they are undecided will vote for. From your computations, it would appear that using an RV model with an allocation of the majority of undecideds to Kerry, you get an estimate that is more pro-Kerry than the count. One possibility, the one you embrace, is that your assumptions are correct, and the difference was due to fraud. Another possibility is that your assumptions were incorrect. After all "Likely Voters" were graded on far more than how recently they had registered (according to your own link: interest in the election; intention to vote; past voting history). And while undecideds might have gone to Kerry, it is something we simply do not know, and is certainly not set in stone.

As for the match - the match of means depends on your assumptions. However, regardless of assumptions, there is a poor statewise match.

I'll say this once more: I think the evidence strongly suggests that 2004 was an unjust election, as was 2000. But I do not consider the exit poll data, or the pre-election data, is evidence that Kerry won a majority of the votes cast. I think that, if anything, the data contra-indicates that conclusion.

____________________

Liz,

You say:
"But eyeballing your own chart, you can see that there is simply no statewise match. Look at Vermont, DC, New Hampshire, New York - exit polls way ahead of the pre-election polls. Then look at North and South Dakota, and Tennessee, all with substantial blueshift relative to the exit poll but a count that was 5 or 6 points "redder" than the pre-election poll."

Wow! You can't be serious. Did you forget that state pre-elect MoEs were around 4% each; and the exit MoEs 2-3%? So 42 out of 50 states with a pre-elect to exit deviation within 4% is not good enough for you? It seems nothing ever is.

Disregard DC, it was an anomaly. The pre-elect had Kerry 78-11, so assigning a 60% share of the other 11% only gave him 84%. as being way off the mark.

So you pick 6 out of 50 states. That's really reaching. What about the other 44? Would you NOT expect some tails at each end of the curve?

This frequency chart sure looks normal to me:
http://www.geocities.com/electionmodel/DEVPreActExitAct_11313_image001.png

And so does the corresponding chart of pre-election and exit poll deviation to the final vote (see the one and two percenters):
http://www.geocities.com/electionmodel/DEVPreActExitAct_5178_image001.png


Your other comment, that the state exit polls were weighted to the pre-elects, is inteersesting. I thought they weighted to prior elections and census data. Can you expound on this? Which pre-elects? All of them? Or just one? Which demographics?

Since the final pre-elects were done just a few days before the election, did Mitofsky take the results and incorporate them into his sample precincts at the last minute? I thought the poll was designed long before the final pre-elects were done. Did he incoporae an undecided factor?

What about the National Exit Poll? Was that also weighted to the national pre-elects?
How so? Did the national pollsters provide, or even ask for, income, party-id, race, gender, military backgound, when decided, who they voted for in 2000?

____________________

Mark,

Were new, young voters (mostly democrats) included in LV polls? How many were even contacted, being that they owned cell phones? Or are you saying that someone who has never voted could still qualify as an LV, even if they say they are likely to vote? What is their voting track record?

Now who led the national pre-elects?
It was Kerry, but it was close enough to call it a virtual tie going in...but remember to allocate the undecided vote.

And don't forget the Bush 48.5% rating on election day.

And that in a study of 155 elections, 82% of the undecideds went for the challenger, just 12% for the incumbent.

And that the 12:22am National Exit poll had Kerry a 5-4 winner of the undecideds.

And that Harris said undecideds usually break 2-1 to 4-1 for the challenger - and they appeared to be breaking to Kerry.


NATIONAL PRE-ELECTION 18-POLL WEIGHTED AVERAGE (before UVA allocation):
UVA = percent of undecided voters allocated to Kerry.

POLL KERRY BUSH OTHER
AVG 47.80 47.14 1.0

Sensitivity analysis of 18 pre-election polls to UVA:
UVA KERRY BUSH
50% 49.83 49.17
55% 50.03 48.97
60% 50.24 48.76
67% 50.52 48.48
75% 50.85 48.16

Compare the above to:
NATIONAL EXIT POLL
(12:22am, 13047 respondents)

CATEG KERRY BUSH
Gender 50.78 48.22
PartyID 50.69 47.50
Voted2k 51.41 47.62

Good enough fit?


NATIONAL PRE-ELECTION 18-POLL SUMMARY:
Kerry won 11, Bush 6, 1 tie
Kerry won 5 of 9 Registered Voter (RV) Polls
and 6 of 9 Likely Voter (LV) Polls

Total Poll Total Weighted Average
Sample Sample MoE KERRY BUSH
Date 26961 Group 0.60% 47.80 47.14

1-Nov Marist 1166 LV 2.87% 50 49
1-Nov Econ 2903 RV 1.82% 50 47
1-Nov TIPP 1284 LV 2.73% 44 47
1-Nov CBS 1125 RV 2.92% 47 48
2-Nov Harris 1509 LV 2.52% 50 47

2-Nov Zogby 1200 LV 2.83% 50 47
31-Oct FOX 1400 RV 2.62% 48 45
31-Oct DemCorp 1018 LV 3.07% 48 47
31-Oct Gallup 1866 RV 2.27% 48 46
31-Oct NBC 1014 LV 3.08% 47 48

31-Oct ABC 3511 RV 1.65% 47 48
30-Oct ARG 1258 LV 2.76% 49 48
30-Oct Pew 2408 RV 2.00% 46 45
29-Oct Nwk 1005 RV 3.09% 44 48
26-Oct ICR 817 RV 3.43% 48 48

24-Oct LAT 1698 RV 2.38% 48 47
21-Oct Time 803 LV 3.46% 46 51
20-Oct AP 976 LV 3.14% 49 46

____________________

Anonymous:

TIA wrote

Wow! You can't be serious. Did you forget that state pre-elect MoEs were around 4% each; and the exit MoEs 2-3%? So 42 out of 50 states with a pre-elect to exit deviation within 4% is not good enough for you? It seems nothing ever is.

As I keep saying, TIA, the MoE is irrelevant to this. The count, the pre-elects and exit polls are all highly correlated with each other, and a lot of the polls are within the MoE of each other and the count (depending on how you calcuate the MoE). What matters is whether there is a trend for polls to be more one way than the other (i.e. overall bias). This is clearly the case for the exit polls, but I am not convinced it is true for the pre-election polls for reasons I have given.

But if you want to know whether the deviations of the exit polls from the count have the same cause as the deviations of the pre-elects from the count (namely, fraud) you need to see if the deviations are correlated. And they aren't. The fact that you can make the pre-elects go the same way as the exits by making certain assumptions regarding models is neither here nor there. What matters is that the two sets of deviations are not correlated with each other

Thanks for your questions. I'll do my best.

I don't know anything more about the pre-election polls than is given in the E-M report, and what we can infer from the fact that the Composite Estimates (which are weighted to the Prior) tended to be closer to the count than the Best Geo (which isn't), i.e. that the Priors must have been in general closer to the count.

In the glossary they say:

    Prior Estimate
    Prior Estimates are based upon pre-election surveys conducted in each state. The Prior
    Estimate is used in combination with the Best Survey Estimate during election day to
    create a Composite Estimate.

As far as I know, the screenshots you have would have reflected weighting to pre-election polls, but I don't specifically know about the National. Certainly your state numbers seem quite close to the Composite Estimates, but I haven't checked.

I do suggest you download the E-M report (link given above) and look at those spreadsheets. They are quite incredibly informative and give both Best Geo and Composite Estimates untainted by vote-returns, as well as the standard errors. Mean state WPEs are also given, as are tables of the degree to which counts from the NEP polls differed from the mean state counts (not a lot).

There is also a glossary at the beginning.

Gotta go to bed.

Cheers

Lizzie

____________________

Mark Lindeman:

"Were new, young voters (mostly democrats) included in LV polls?"

Yes, as I assume you already know. I mean, you do actually look at some of these polls, don't you? Don't you?

Or do you really feel comfortable just making stuff up as you go?

Really, how can you harangue people about this subject for two years and know so little about it? Wow.

____________________

Mark,
You started this. You are baiting me, as usual. So I must now respond.

Your comments consistently avoid or mistate the facts I have presented. In fact, your non-rebuttals have only served to enhance the analyses.

Since YOU have not specifically rebutted any of the data or analyses I have presented here, this is your opportunity. Take any of my comments (assumptions or data or analysis) and rebut them. One by one.

Go right ahead. Do it. Right here. Right now.

You said upthread that I should be "embarrassed" by pointing out that Pew had TWO final polls, an RV and an LV. How lame. What is it about those RV polls which showed Kerry winning that you object to? Why should I be "embarrassed" for presenting both RV and LV polls to illustrate my point?

YOU should be embarrassed for making that statement.

I'm NOT Salon's Farhad Manjoo, who relied on your "expertise" for his hit job on RFK, Jr. I know something about mathematics. Farhad obviously has no quantitative skills. Did you tell him to state that the 3% discrepancy between the exit poll and vote (Kerry 51-48% vs. Bush 51-48%) was not significant? The NEP had a 1% MoE, according to Mitofsky himself. SIX standard deviations from the mean is not significant?

If you didn't tell him to write that, why didn't you correct him after reviewing his final draft? You did review it, didn't you? After all, he was using your information.

Now, back to your comment. Take a look at who the 18-29 group voted for. According to the NEP, they voted for Kerry. At the 12:22am timeline and the Final, which we both know was matched to the vote. I believe the 12:22am timeline was closer to the correct percentage.

Do you not even believe the Final? If you don't, what are your numbers? And where did they come from?
_____________________________________

National Exit Poll
12:22am, Nov.3,2004
(13047 respondents)
AGE Votes Weight Kerry Bush Other
18-29 20.56 17% 56% 42% 1%
30-44 35.08 29% 48% 49% 2%
45-59 36.28 30% 51% 47% 1%
Over 60 29.03 24% 48% 51% 0%

Total 100% 50.26% 47.69% 1.05%
120.95 61.40 58.26 1.28


National Exit Poll
2:04pm Nov.3,2004
(13660 respondents)

Weight Kerry Bush Other Total
17% 54% 45% 0% 99%
29% 46% 53% 1% 100%
30% 48% 51% 0% 99%
24% 46% 54% 0% 100%

100% 47.96% 51.28% 0.29% 99.53%
121.60 58.59 62.65 0.35 121.60

____________________

Mark Lindeman:

Actually, you can hardly imagine how much time I am wasting over here attempting to be minimally polite. Your previous question speaks for itself.

The problem isn't that you present both RV and LV numbers -- it's that you reject the LV numbers because you don't like them. And in order to rationalize that, you make up how the LV modelers handle new registrants instead of looking it up. Palpable laziness does not help the cause, really it doesn't. I expect more.

You think Manjoo said that the U.S. exit poll discrepancy was not statistically significant? Evidence, please.

____________________

Mark, you quoted me:
"Were new, young voters (mostly democrats) included in LV polls?"

And then you responded:
Yes, as I assume you already know. I mean, you do actually look at some of these polls, don't you? Don't you?

You totally missed my point.

Young (1-29) newly registered voters were more likely to be considered as RVs rather than LVs, because
1) they never voted
2) they have a much lower turnout history than other age groups,
3) Most voters in the 18-29 range are democrats.
4) most young voters are usually out of the house, rely on cell phones, and therefore they are less likely to be polled.
5) in 2004, the democrats registered significantly more NEW voters (especially young voters) than the republicans.

Would you classify new voters as LVs or RVs?

Now do agree that in 2004, RV polls would include more democrats than LV polls?

And that's why I am not embarrassed by quoting both Pew polls.

Comprende, amigo?

____________________

Typo fix:

Young (18-29) newly registered voters were more..

____________________

I refer you to this post rebutting Manjoo on Daily Kos, as I do:

http://www.dailykos.com/story/2006/6/3/224759/7766

snip

This part of Manjoo's analysis ends with a statement that is so beyond my comprehension, I fail to see how it could be anything but and error, so I'll just quote it, and let it stand by itself:

"Mitofsky's final national poll put Kerry at 51 percent and Bush with 48 percent. Compare this to the actual result, which had Bush at 51 percent and Kerry with 48 percent. The difference is not that significant".

snip

____________________

Mark Lindeman:

TIA, just to note: no one ever disputed that RV polls had more Dems than LVs (what, are you joking?), or even that new registrants were less likely to be considered LVs (although you haven't even documented that). You can keep leaping from argument to argument; I don't have time to chase you. If at some point you become interested in persuading non-believers, you will have to work harder.

You've quoted Manjoo out of context (he was comparing the exit poll discrepancies in the U.S. and in Ukraine), and the quotation still doesn't support your characterization. There is no reasonable way to render "not that significant" as "not statistically significant."

My TIA FAQ should be out soon, but in the meantime, you could work your way through every argument in my AAPOR paper. I will not hold my breath.

Bye.

____________________

Pat R.:

In a nation that specifies voting as the method of hiring public officials, the presumption is that all votes count.

If all votes count, it is up to the municipality to provide the means and mechanics to insure that votes count - in that precinct or in another.

Where America has a banking system that can assure efficient transfers of money so that people can gain access to their money from anywhere, it is illegitimate to imagine that a person can vote and that vote would not be counted in the place it belongs.

Shifting the responsibility onto the voter rather than keeping it within the party or municipality that makes voting possible is inherently a violation of voting rights.

The act of not counting a vote is itself a Constitutional violation worthy of severe penalties by ignoring the presumption of voting as the status quo, not the exception.

No processes need be validated that penalize the voter for voting in a country where voting is considered an automatic right.

____________________

it is sure that without a controlled experiment that certainly to make the difference between both orderings on a survey with an enormous sample size, won't be able to help to know for certain.

____________________

The survey clearly stated that thing was better when Mr Bush was the president. It seems that much more believe and feel things is worse that it was before.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR