August 13, 2008
By Guest Pollster
Today's guest pollster contribution comes from Humphrey Taylor, who has served as chairman of The Harris Poll, a service of Harris Interactive, since 1994.
When I started work in market research, I spent the first month learning to interview people face to face, in central Scotland. It was a great experience. At the first house, a woman answered the door and, as I nervously explained that I wanted to interview her, she shut the door in my face. My supervisor, a cheerful, middle-aged woman, rang the bell again and, all smiles and self-confidence, easily completed the interview. The same thing happened at the next house, and my self-confidence hit rock bottom.
At the third house, to my enormous relief, I actually managed to complete the interview with an elderly Scottish lady. But as I thanked her, she said, with a twinkle in her eye, "Och, surely you don't believe all the things the folks tell you, do you?" This may have been one of the most valuable comments anyone has ever made to me about survey research, and I have remembered it many times over my working life.
The sad truth is that all too often we researchers naively accept what respondents tell us without questioning if it is really "true." The following exercise explains part of the problem. Ask someone which candidate or party they prefer and they will usually give you an answer (and, by the way, it will probably be true). But then ask "Why?" and the conversation will probably go something like this:
"Because of his/her/their policies."
"Which policies specifically?"
"His. . . um.. economic (or Iraq, health care, etc.) policies."
"What are his economic policies?"
Push harder and you will probably find that this voter really doesn't know what the candidate's economic (or most other) policy proposals really are. But this does not stop people from having a strong preference for one candidate over another, or believing that they would handle the issue mentioned better than their opponents.
One model of how voters choose candidates is that they are like juries. They listen to the candidates and carefully consider their policy proposals before deciding which way to vote. Unfortunately this theory is almost never true.
There are many reasons why it is so difficult to understand people's motives. One is that most people don't understand themselves and often rationalize their attitudes and behavior. Sometimes they surely deceive themselves and sometimes they knowingly bend the truth or tell outright lies. There is a growing body of literature that documents the unreliability of replies given to interviewers where there is a "socially desirable" answer. Large numbers of people lie in telephone and in-person surveys about whether they believe in God, go to church, give money to charity, clean their teeth regularly, drive over the speed limit or drink alcohol. Many people who do not vote claim that they do. And the number of people who say they voted for a sitting president tends to go up when he is very popular and down when his ratings fall.
Another problem is that people give inaccurate answers not because they are lying but because their memory is imperfect. And many people's honest predictions of their own future behavior are notoriously inaccurate.
When it comes to voting, there are many factors which influence voters' preferences, most of which they are often unaware of. While the candidates' positions on the issues (or voters' perceptions, which may be inaccurate, of their positions) are an important factor, there are several more powerful ones.
Voters often explain their votes based on the candidates' track record. Obviously these are important but voters' perceptions of politicians' track records vary greatly depending on their political and ideological views. Some voters think President Bush should have been impeached. Others think history will show him to have been a great president. While perceptions of politicians' record often influence voter preferences, the reverse is also true -- that voter preferences have a huge impact on perceptions of their track records.
So what other factors have a big impact on voting behavior? One is family, friends and people they work with. Most people vote the same way as most of the people they like and socialize with.
A candidate's voice, looks, style and rhetoric are all enormously important. Franklin Roosevelt, the only president to be elected four times, was the perfect candidate -- good looking, with a beautiful voice, a commanding presence and a wonderful way with words. But if you had asked people why they voted for him they would probably have referred to his policies or his track record and what they thought he would do.
One of the reasons Ronald Reagan was so successful as a politician was that he, and his pollster Dick Wirthlin, understood that "values" were more important than "issues." Reagan mastered the art of persuading people he shared their values, so that many people who did not support his positions on some key issues voted for him anyway. In addition he, and his aide Michael Deaver were masters of the photo-op, casting Reagan in great settings that made him look very presidential. But voters would not tell you this influenced their votes
There are many other favors that influence candidate preferences and voting behavior that are rarely mentioned by voters. Political advertising has a huge impact but few people believe, or tell surveyors, that they are influenced by advertising. The media voters are exposed to matter a lot. Those who read the editorials in the Wall Street Journal or The New York Times have their opinions shaped, or reinforced, by what they read. And those who watch Fox News get a very different world view than those who watch other television stations. As dictators and media moguls know well, the media's ability to shape public attitudes is very powerful.
So next time a voter tells you his vote is determined by the candidates' positions on the issues, treat this with a large dose of salt.
By Guest Pollster on August 13, 2008 4:13 PM
| Permalink
| Comments (0)
| TrackBacks (0)
August 6, 2008
By Guest Pollster
Nick Panagakis is president of Market Shares Corporation, a marketing and public opinion research firm headquartered in Arlington Heights, Ill.
This post is the forth installment of a dialogue between pollsters David Moore and Nick Panagakis about the best way to measure and report how many voters are "undecided." See their earlier installments here, here and here.
I agree with much of what David Moore says in his response, including percentage undecided that seems too low as is currently being reported. Where we differ is on terminology. The potential for mind-changing is a lot less than you think.
Yes we are "interested in portraying what the electorate is thinking today". Now that general election national polling is underway, we will be interested in finding (needless to say) whether voters did change their minds about the candidate(s) since the last poll asking how they would vote "if the election were held today".
My issue is about reporting results with low conventional undecideds followed by a large number in the 20%+ range who could still change their minds. It's enough to give readers and viewers whiplash.
In my last post on this subject I hypothesized that such high numbers are not "indecision" as implied by "could change their minds". I said some voters willing to decide on a candidate in a poll won't rule out the possibility that some incident or candidate disclosure, however remote, could lead them to vote otherwise.
The ABC Polling Unit provides some validation of this. Their polls have been asking this question of decided voters since 2004: "Would you definitely vote for ___ or is there a chance you could change your mind and vote for someone else?" This has been asked three times since May this year and eight times in 2004, from June 20 to September 26. This year, "could change your mind" has ranged from 25% to 29%, similar to response levels seen in current polls, dissimilar wording not withstanding. In 2004, "could change your mind" was 28% in the June 20 poll then steadily declined to 16% in late September.
But unlike other polls, ABC then probed potential mind-changers by asking "Is there a good chance you'll change your mind, or would you say it's pretty unlikely?" So far this year, about half say "pretty unlikely" as did respondents in June, 2004 polls. July to September 2004 showed another pattern. 'Pretty unlikely" voters began to consistently outnumber "good chance" of mind-changing voters by a ratio of 2 to 1. This could mean that two-thirds of possible mind-changing voters in current polls, if asked their chances of doing so, would rate their chances as pretty remote. Should mind-changing as currently being presented be part of any story when the chances of doing so are so slim? I don't think so. I prefer the ABC qualifier.
Another thought. Shouldn't there be some analysis to validate such high could change their mind numbers? The analysis could compare poll stated undecideds with "could change their mind" levels with actual candidate vote preference changes from poll to poll and to election outcomes.
Another subject. David mentioned the recent CBS poll. According to their release they had 12% undecided which seems reasonable to me. If you go to pollster.com's national summary you will find many polls with much lower undecideds. However, half of Gallup's higher undecideds shown there are actually vote for "neither" which should not be combined with undecideds in that table. Click the Gallup links. Moreover, "neither" response is not very meaningful. It would be more precise to replace it with "vote for other" and "won't vote" with non-voters excluded from the base for calculation of voter percentages. All for now.
By Guest Pollster on August 6, 2008 4:55 PM
| Permalink
| Comments (0)
| TrackBacks (0)
July 30, 2008
By Guest Pollster
Nick Panagakis is president of Market Shares Corporation, a marketing and public opinion research firm headquartered in Mt. Prospect, Ill.
This is in response to David Moore's July 25th column about use of a broader measure of voter indecision. For the first time, I also asked a similar question before the February 5th Illinois primary but am now having doubts about it's usefulness.
In the final CNN/University of New Hampshire primary poll, over 90% of voters stated their preference for a candidate in the commonly used "if the election were held today" forced choice question. That poll had Obama up by 9%. But Clinton won by 2.6 points. The candidate estimate error was 5.8 points, that means 5.8% high on Obama and 5.8% low on Clinton, near the average of all NH polls. When voters in that poll were asked if they were definite, leaning, or "still trying to decide", some 21% said still trying to decide which was the subject of Moore's blog.
Among the 21% who were "still trying to decide", that could mean 6% of all voters switched from Obama to Clinton or, a net 6% more voters switched from Obama to Clinton than from Clinton to Obama. The 21% more than covers such movement.
Other New Hampshire polls showed comparable numbers: Gallup's "could change mind" and in late December the LA Times' "might end up voting for someone else" both yielded 27%. I checked polls in other states that asked similar questions of decided voters and show comparable high percentages with no evidence that such mind-changing ever took palace.
My first issue is that the forced choice "if the election were held today" question historically comes close to the actual outcome, even though some voters may not have reached final closure when asked. I wouldn't call this "indecision" after so many could decide in response to the standard question. I believe it means some voters who are wiling to decide on a candidate in a poll won't rule out the possibility that some incident or disclosure, between now and election day, could lead them to vote otherwise. Isn't that what campaigns including negative elements are all about? This response is more conditional, perhaps remote, depending on unknown future events, not indecision. If it were indecision, a lot more polls than New Hampshire would have been be off the mark this Spring. In the post New Hampshire period, I cringed when I saw such numbers being reported. I think they de-values polls. There must be some better way of reporting these findings rather than "candidate A is up by 9 points - but 30% could change their minds".
During the week preceding the February 5th Illinois primary, our Chicago Tribune poll showed Obama ahead by 31 points in that primary, very close to the actual outcome. Our poll also got a similar number just days before election day - 24% of decided and leaners said they could "still change their minds". Could it be that a few days before any election, somewhere around 20%-25% of voters in all polls always say they could still change but most never do? Based on the Illinois outcome, not many minds were changed as is the case in most polls. To me, it seems that how voters would decide today has served us pretty well with some exceptions such as New Hampshire. (The question read: "Between now and next Tuesday, is there some chance that you could still change your mind about voting for this candidate...or have you definitely made up your mind?")
Re-calculating our Illinois Democratic poll numbers to combine possible mind-changers with undecideds as Moore did with the New Hampshire poll resulted in: Obama 44%, Clinton 16%, Others 1%, and 39% undecided. (The apparent reason for 39% here was an increase in conventional undecideds due to Edwards dropping out the day before interviewing began. Edwards did have 15% support in Illinois in a poll conducted a few days earlier by St. Louis Post-Dispatch,/KMOV-TV poll.) According to MSNBC, the NEP Illinois exit poll found 19% of voters who said they decided in the last 3 days, the period after we completed interviewing, close to our conventional undecideds. But the recalculated 39% undecided above that included voters who could "still change their mind" is twice as high as the 19% of voters NEP found deciding on a candidate during the 3 days before that election.
In the Illinois Republican primary, 36% of voters and leaners said they could change their minds. McCain was ahead in the poll by 23 points and went on to win by 19 points, a 2-point error on candidate estimates. Moore did not include a comparable number for the New Hampshire Republican primary but all polls matched the outcome.
In conclusion, perhaps in the New Hampshire Democratic primary this year such mind-changing took place. The state has always been a minefield for pollsters. The challenge for pollsters was mostly situational. This was a fluid situation, akin to trying to catch a falling knife. The campaign period was compressed, shortest-ever in New Hampshire, only 5 days after Obama's Iowa upset. Obama was described as over-confidant. Clinton perceived as a victim by some.
There were methodological challenges. Turnout that this year turned out to be historically high (a forewarning for us pollsters in later states). Only 52% of voters in the New Hampshire Democratic primary were registered Democrats according to the exit poll and 19% were first-time primary voters, a challenge for likely voter screening. According to one pollster, their best estimate of the New Hampshire outcome was based on all registered voters; i.e., no sample reduction at all for likely voters. The final chapter on this election has not yet been written. Neither has the value of routinely reporting that 25% or more of voters are undecided.
By Guest Pollster on July 30, 2008 5:51 PM
| Permalink
| Comments (1)
| TrackBacks (0)
June 27, 2008
By Guest Pollster
Today's Guest Pollster article comes from Thomas Riehle, a Partner of RT Strategies.
If it were a different month on the calendar - say, October - the Obama campaign might be concerned to see that the groups most likely to be truly undecided, not leaning even a little toward Obama or McCain, comprise some voters he must be counting on:
- Women (21% undecided), women ages 40-64 (24%), women ages 65 and older (25%), women with less than a 4-year college degree (24%), and
- Registered voters in the Northeast (20%) and Great Lakes (20%).
Moreover, no one at the Obama campaign can be happy to see that the vote is currently tied among women with a college degree or more (43% Obama - 42% McCain) - highly educated women having become one of the most reliably Democratic groups in the electorate. Obama may start to win back support from among the relatively large group of McCain supporters currently to be found among women who voted for Clinton in Democratic primaries or caucuses (25% now support McCain), college-educated Clinton primary voters (28% McCain), moderate or conservative Clinton primary voters (23%) - but right now, those are some significant defections.
The calendar says June, not October, and undecided voters eventually will make up their minds. All in good time. For now, the McCain camp and Obama's camp are looking for indications of subtle trends moving in the early stages of the general election. For that kind of tracking, many campaigns use a tool called the "Hierarchical Vote." It divides support into 7 categories, from most pro-McCain to most pro-Obama, and tracks movement from one category to another across the 7 categories of support. Tracking changes month-to-month in the Hierarchical Vote overall among all voters (and within 50 or more subgroups) gives a campaign the insight needed to focus resources on the groups who are ready to move right now.
On the Cook Political Report website, you will find posted a Hierarchical Vote analysis from the past four Cook Political Report / RT Strategies polls (March through June). I hope you find it a unique and useful way to delve under the familiar topline vote totals and see what's really going on as we approach Independence Day. We'll keep updating this Hierarchical Vote table as Election Day approaches - just as the campaigns will do. Please let me know what you think and what you learn in reviewing these results.
By Guest Pollster on June 27, 2008 1:31 PM
| Permalink
| Comments (14)
| TrackBacks (0)
May 28, 2008
By Guest Pollster
Today's Guest Pollster's column comes from Robert M. Eisinger, a political science professor at Lewis & Clark College and the author of The Evolution of Presidential Polling (Cambridge U. Press).
The Obama "phenomenon" is a product of many things, most notably a superbly smart candidate and a sharp, disciplined campaign team, both of whom clearly articulated a resonating message that mobilized voters. What we don't know if how many of those supporters galvanized around Senator Obama simply because he is not Senator Clinton. This is not to say that Senator Clinton is without fans. To the contrary - she has many. They are devoted and dedicated, imparting the kind of loyalty that any political candidate would desire.
But as evidenced by the reaction to her recent mentioning of Robert Kennedy's assassination, Senator Clinton appears to be a lightning rod - people are either repelled or attracted to her.
Arguments for an Obama-Clinton dream ticket suggest that Senator Clinton's keen intelligence, legislative acumen and support among Democrats outweigh her negatives. However reasoned this claim is, it is potentially flawed in a critical way currently understudied by the public opinion and political cognoscenti. The problem is that we do not know enough about her positives and negatives, especially among voting Democrats and swing voters.
By itself, the "favorability question" is crude and insufficient indicator of likeability. It does not claim to measure the intensity underlying that favorability or lack of favorability. For example, a May 23, 2008 Newsweek poll asks, "Who would you MOST like to be nominated as the Democratic Party's presidential candidate this year...Hillary Clinton (or) Barack Obama (choices rotated)?" "Do you support (INSERT CHOICE) strongly or only moderately?" The poll then asks, "We'd like your overall opinion of the presidential candidates. As I read each name, please tell me if you have a favorable or unfavorable opinion of this person - or if you have never heard of them before this interview. What about (INSERT - READ AND RANDOMIZE). Do you have a favorable or unfavorable opinion of him (or her)?"
Favorability questions specifically asked about Hillary Clinton in the past have been worded in numerous, thoughtful ways, including the following: 1
CBS News/New York Times: Is your opinion of Hillary Clinton favorable, not favorable, undecided, or haven't you heard enough about Hillary Clinton yet to have an opinion?
Gallup/USA Today/CNN: I'd like some overall opinion of some people in the news. In general, do you have a favorable or unfavorable opinion of Hillary Clinton?
Yankelovich/Time/CNN: Please tell me whether you have generally favorable or generally unfavorable impressions of [Hillary Clinton], or whether or not you are familiar enough with [Hillary Clinton] to say one way or another.
Each of these questions is carefully written, but they do not capture the intensity that may lie beneath the answer. In fact the Yankelovich/Time/CNN question employs the phrases "generally favorable" and "generally unfavorable", allowing the respondent to articulate her overall impression, but in doing so, diffuses the potential passion or force embedded within that answer.
If respondents were asked to place their favorability/un-favorability on a seven point scale, then one might get a better sense of the potential polarizing nature of the answer, or put another way, the strength of that favorability and its opposite. Such an option is costly in that it requires an additional question to be asked, and more nuanced data analysis.
Anecdotal conversations in the blogosphere, in the taxi cab and the around the water cooler - reveal that many citizens - men and women, Democrats, Republicans and Independents - have a palpable and deep disdain for Senator Clinton. Different blog, cab and water cooler discourses tell us that Senator Clinton is revered. Scholars of public opinion and savvy journalists are appropriately suspicious of these unrepresentative remarks. Sure, the plural of anecdote is data, but we wonder if selection bias (i.e., we surround myself with like-minded folk; we listen more carefully only to extreme answers) taints our perspective and analysis.
The Obama campaign should be privately measuring the favorability intensity for all prospective Vice Presidential nominees. If not, then they are avoiding a datum that may be critical to their electoral success. Media polls should explore this question as well; given the dearth of the intensity question, the answers will undoubtedly surprise us. //END
1 From Barry C. Burden and Anthony Mughan, "Public Opinion and Hillary Rodham Clinton," Public Opinion Quarterly, Vol. 63, No. 2 (Summer 1999), 237-250.
By Guest Pollster on May 28, 2008 3:20 PM
| Permalink
| Comments (11)
| TrackBacks (0)
May 23, 2008
By Guest Pollster
Today's Guest Pollster contribution comes from Robert M. Eisinger, a political science professor at Lewis & Clark College and the author of The Evolution of Presidential Polling (Cambridge University Press).
There are few things more dangerous to sensationalized journalism than when anyone over-analyzes poll data. A recent Quinnipiac Poll shows Senator Clinton defeating Senator McCain in Ohio and Florida, but Senator Obama losing such head-to-head match-ups against Senator McCain. A SurveyUSA poll shows similar results in Missouri and North Carolina. Clinton defeats McCain, but Obama does not. These polls, it is argued, are worrisome for the Obama campaign, and serve as evidence among some Clinton supporters that she is a stronger candidate in swing states.
Beware. Poll answers, regardless of the question, must be placed in some context. The absence of at least one follow-up question may have yielded an interesting context from which to interpret the head-to-head answers provided. Imagine, for example, that the pollsters asked the following question:
"Imagine that Senator Obama eventually becomes the Democratic presidential nominee, and Senator Clinton enthusiastically campaigns for him. If the 2008 election for President were being held today, and the candidates were Barack Obama the Democrat and John McCain the Republican, for whom would you vote?"
One could even imagine tweaking the question by revising "Senator Clinton," "the Clinton campaign," "Senator Obama," and "the Obama campaign." Such questions are reasonable one to ask, especially when one is reminded that both Senators Obama and Clinton have stated that they would endorse the Democratic nominee. There is good reason to believe that Senator Clinton would be magnanimous and enthusiastically support the Democratic ticket. Similarly, there is no reason to conclude that the inclusion of this question would necessarily result in poll numbers that would greatly assist Senator Obama; it is quite conceivable that some of Senator Clinton's supporters do not want to a President Obama under any circumstance, and that Independent voters may be turned off by Senator Clinton's endorsement of anyone.
No doubt the Obama campaign is not rejoicing after reading the poll data. They would prefer numbers that show Senator Obama defeating Senator McCain in all states at all times. Senator Obama wants to win the swing states, and right now, Florida, Ohio, North Carolina and Missouri are not securely in the Democratic camp.
But there is something noticeably absent about asking about a viable scenario in which the Democratic presidential candidates, especially Senator Clinton, unite behind the winner, even if the nominee is Senator Obama.
[Typos corrected]
By Guest Pollster on May 23, 2008 3:29 PM
| Permalink
| Comments (3)
| TrackBacks (0)
May 6, 2008
By Guest Pollster
Humphrey Taylor has served as chairman of The Harris Poll, a service of Harris Interactive, since 1994.
Thanks to Doug Usher for his contribution to the debate on the panel-based online methodology for political polling. I am glad to see that he acknowledges the value and viability of this method for national polls. But I am puzzled as to why he thinks this method will not work in Congressional races (at least I think that is what he is saying). He writes of the "sour spot" based on the fact that political polls need to reach "a narrow population for which pollsters do not have well defined web contact information".
I assume he means by this that we cannot sample a geographic area because we do not know where people live. Of course we can, and we do this easily. We and others who have large panels, know the states in which people live, so that takes care of senate races.
What about congressional districts?
Some panels also have zip code information, and those that do not can screen for it. In so far as some zip codes straddle the boundary with another district we can screen for streets or even addresses if necessary. And of course this problem is the same ,or possibly worse, for RDD telephone samples ,as telephone exchanges may also straddle the boundaries between districts. Furthermore many people now take their telephone numbers with them when they move from one district to another.
My comment that was quoted by Doug Usher was taken out of context. I certainly believe that online political polling methods are "the wave of the future". My mention of cell phones was specifically in reference to telephone surveys of people aged 18 to 29. I am sorry if that was not clear.
By Guest Pollster on May 6, 2008 12:09 PM
| Permalink
| Comments (0)
| TrackBacks (0)
May 1, 2008
By Guest Pollster
Berwood Yost is Director of The Floyd Institute's Center for Opinion Research at Franklin & Marshall College. Kirk Miller is B.F. Fackenthal Professor of Biology and Senior Research Fellow at The Floyd Institute's Center for Opinion Research.
The 2008 Democratic presidential primary on April 22 put Pennsylvania in the national spotlight for a long six weeks. Members of the media followed the candidates into the Keystone State intending to learn more about its people and its politics. Not far behind the media came the pollsters--some media even brought their own pollsters. Pennsylvania voters were besieged by pollsters in unprecedented numbers. There were 39 publicly released surveys, which included more than 30,000 interviews with the state's voters, during only the last three weeks of the campaign. This is a tremendous increase in polling activity compared to the 26 polls released in the final three weeks of the 2004 presidential campaign in Pennsylvania or the 15 released during the final three weeks of the 2006 Senate campaign.
Taken together, the pollsters who pestered Pennsylvanians did an adequate job of predicting the final outcome: 36 of the 39 polls in April predicted a Clinton victory and the three outliers were all conducted by the same polling organization. We agree with Charles Franklin's assessment that the aggregate performance of the Pennsylvania pollsters was good. Figure 1 is a frequency distribution of the predictive accuracy of the 39 public polls released in Pennsylvania. It shows that there was a slight bias in the polling estimates toward Barack Obama (meaning the polls in Pennsylvania underestimated Hillary Clinton's margin of victory), but that this bias was small and, according to the exit polls, not surprising because late deciding voters moved in larger proportions toward Clinton.
Figure 1 Frequency Distribution of Predictive Accuracy
Some individual pollsters faired much better than others in the accuracy of their estimates. Figure 2 shows the predictive accuracy and corresponding confidence interval for each of the 39 polls conducted between April 1 and 22 in Pennsylvania, arranged by the number of days prior to the primary the survey was completed. Those pollsters who produced a biased estimate, meaning the confidence interval for their estimate did not overlap zero, are labeled in Figure 2. Three of the four polls conducted by Public Policy Polling (PPP) were biased and all were biased toward Obama. Two of the three polls conducted by American Research Group (ARG) were biased and one of SurveyUSA's three polls showed bias. One ARG poll showed that Clinton and Obama were tied; the other, seven days later, showed Senator Clinton ahead by 20 points. The SurveyUSA poll that missed also showed Senator Clinton ahead by 20 points. The measure of predictive accuracy we used shows that the pollsters' final estimates were mostly in line with the final election results.
Figure 2 Predictive Accuracy of Individual Polls by Date of Poll
The misses identified in Figure 2 are not related to sample size. Four of the surveys that missed had four of the eight largest samples; the other two that missed had sample sizes that were only slightly below the median size. There is a relationship in these analyses, as one would expect, between sample size and the widths of the confidence intervals, but there is no relationship between sample size, width of the confidence interval, and the likelihood that a survey was biased. We don't know what methodological choices matter most in producing unbiased polls without further examination of the methodological choices the pollsters make. Some might conclude that pollsters who use inter-active voice response (IVR) technology to collect data are more prone to bias because two of the three pollsters who produced biased estimates use IVR, but not all IVR pollsters produced biased results.
Another interesting question we tried to answer is whether the polls converged on the end result as election day approached. Depending on the method used, the answer is a qualified, "slightly." Figure 3 shows the predictive accuracy of each poll as a function of days before the Pennsylvania primary. The trend line fitted to the figure is produced by a LOWESS iterative locally weighted least squares regression. The red dots identify the six biased polls noted earlier. The curve indicates that the polls began to converge until about two weeks prior to the election, that they remained relatively constant for about a one-week period, and then began to converge again over the final days of the campaign. If the six biased polls are removed from the analysis, the convergence is not dramatically improved.
Figure 3 Predictive Accuracy of Individual Polls by Date of Poll with Fitted Regression Line
Measuring Predictive Accuracy
We used the measure of predictive accuracy developed by Martin, Traugott and Kennedy (2005) A Review and Proposal for a new Measure of Poll Accuracy. Public Opinion Quarterly, Volume 69 (3): 342 - 369. Their method compares the ratio of the estimated percent of voters voting for each candidate to the ratio of the final vote tally for each. The natural log of this odds ratio (ln odds) is used because of its favorable statistical properties and the ease of calculating confidence intervals for each estimate. The confidence interval for a poll that reasonably predicts the final outcome of the primary election will overlap zero. Senator Clinton's votes or projected votes were the numerators in all the ratios we calculated so negative values for ln odds represent an overestimate in favor of Senator Obama and positive values represent an overestimate in favor of Senator Clinton. According to this measure, a poll is biased if its confidence interval does not overlap zero. The polling results used in this analysis were taken from Pollster.com.
By Guest Pollster on May 1, 2008 3:30 PM
| Permalink
| Comments (1)
| TrackBacks (0)
By Guest Pollster
Douglas Usher is the Senior Vice President of Widmeyer Communications and formerly Vice President at the Democratic polling firm, The Mellman Group.
"The survey research and marketing industries need to recognize that the Internet and cellphones, not landlines, are likely to be the wave of the future." So says Humphrey Taylor, chairman of the Harris Poll.
I met Humphrey Taylor once - in 1999. He pitched Harris Online services to the Democratic polling firm where I worked, and his team said that telephone surveys in politics would likely be replaced by web surveys after that election cycle.
Was he right about political polling? Hardly - in fact, he couldn't have been more wrong. Let me make this as clear as possible: no professional political pollster on either side of the aisle has ever used web-based surveys for quantitative research in their campaign practice.
And as any pollster.com reader understands - and all serious consumers of political polling know - you can count on one hand the number of public pollsters using online methodology for political polls. Even John Zogby, who claims that his firm has "since the mid-90's... utilized the Internet as a means of providing the public with instant access to the day's best public opinion research," has like most pollsters used telephone polling this cycle.
Internet polling is a growing industry. I use it all the time for my clients - indeed, it rules many aspects of consumer research. So, why the disconnect for politics?
Because quantitative political research for nearly all levels of American politics hits the "sour spot" of internet research.
Let me explain.
Internet-based research is perfectly suited for certain types of public opinion research:
- Qualitative research: in-depth, group level research designed to evaluate reactions to specific ideas, issues, and stimuli (like campaign ads) - research which provides rich feedback, but is not projectable on the population at large. The internet provides a (virtually) limitless pool of volunteers that will provide quick feedback about a candidate, product, print or television ad. It provides a reasonable - and often less expensive - alternative to focus groups, without the travel.
- Quantitative research among broad populations: For the broadest audiences - "adults" nationally, "likely voters" nationally, and "likely voters" in some states - internet research can provide a reasonable (and again, less expensive) alternative to telephone polling. The opt-in panels that internet research vendors build - if properly cleansed and refreshed on a regular basis - have been demonstrated to be reliable proxies to telephone research for point-in-time quantitative measurement.
- Quantitative research among narrow populations where e-mail contact is previously established: This type of research includes organization membership research, or a survey of loyal customers, or an internal corporate survey. One of the fallacies spread by those who sell public opinion research services on the internet is that because people are on the web, they are reachable on-line. But unless someone provides you (or an organization) with their e-mail address, it is nearly impossible to find them. However, for internal organization research, internet research conducted of a complete (or near-complete) population by e-mail has become an excellent alternative to phone surveys.
These three types of research describe most of the public opinion research for which clients pay money - hence, the internet has become a valuable research tool.
And qualitative public opinion research is well-suited for the internet (finally, the end of notoriously unreliable mall-intercepts!)
However, quantitative political public opinion research -- polling -- hits the internet's "sour spot" because it requires reaching a narrow population for which pollsters do not have well-defined web contact information.
How well do you think Harris Interactive's national panel maps on to likely voters in New York's 26th Congressional District? If you were polling Indiana's primary, would you feel comfortable that the list of e-mails that you bought from a vendor actually contained properly registered voters in the state with past primary vote history?
Some internet survey vendors claim that they have representative general election statewide panels. This may be true - but how many times can you go back to that panel before you exhaust it? Pollsters in competitive races will track data for 30 days or more - well beyond the capacity of internet vendors in even the largest state.
It's not because political pollsters are "old-fashioned" that they don't conduct web-based quantitative research - it's because there is no reliable way to reach their candidates' electorates online in a way that meets even a modest level of methodological rigor.
None of this is to discount concerns about telephone polling - ever-lower response rates, and caller-ID and cell-phone only households that makes reaching people on the phone more difficult than ever.
But, for political polling, internet-based research has not proven to be the panacea once (and continually) promised.
UPDATE: Humphrey Taylor responds.
By Guest Pollster on May 1, 2008 3:12 PM
| Permalink
| Comments (1)
| TrackBacks (0)
April 1, 2008
By Guest Pollster
Yesterday, in a burst of blogger exuberance, I posted some charts emailed by my long ago employer Harrison Hickman, the Democratic pollster now associated with the firm Global Strategy Group who also conducted surveys earlier this year for John Edwards. I gave Hickman and his associate credit for the charts but then provided my own interpretation, comments I subsequently qualified. This morning, I did what I should have done in the first place, which is offer Hickman the opportunity to describe the charts in his own words. Harrison's summary follows below.
--Mark Blumenthal
The initial (and only) purpose of the line charts Ben Margolis and I sent Mark yesterday seems to have been obscured by our failure to provide explanation with the charts and some of the verbal vines their publication stimulated ("Day-of-Week Effect in Gallup Daily?"). Hopefully I can provide something of a corrective for the former.
1. We submitted the charts without explanation but with obvious doubts about their significance, statistical and otherwise. The subject line of my original e-mail to Mark was "spurious or what?"
2. The point of the exercise was to note that all the hoopla about Obama or Clinton being ahead or behind by more or less at a specific point was ignoring a persistent pattern in the data. (The time period covered was since the departure of the Sainted Senator Edwards.) The point was that the hoopla was misguided, not that the pattern itself is all-telling. We certainly never intended to suggest that particular changes could be associated with specific days or that there was any iron law of anything at work. Our message: if you don't like the results today, wait a couple of days. If you do, it might be wise to exercise some restraint. In that vein, Mark is correct in urging caution about reading too much into day-to-day changes. I would urge similar caution in the interpretation of two techniques under discussion here.
3. The rolling average technique was developed to introduce a cost-effective way to report opinion data more or less continually in critical points of a campaign, and there are a variety of different ways to calculate those averages. But it is important to note that the "smoothing" artifact is the reason the technique is useful, not the reason it is misleading. An on-going series of one-day polls would be more misleading for campaign professionals and poll consumers than rolling averages.
4. Perhaps the most important statistical point to understand about these types of polls is that a sample is not a sample until it is completed. Before its completion, a sample is not "random" even in the colloquial sense of the term. It is for this reason that no one should mistake the partial results of stand-alone samples as precise, no matter how extensively those partial data are weighted. This is particularly important to remember when confronted with early wave results of election day polls (exit polls).
5. One should be mindful of but not obsessed with any particular statistical test. Estimation error is the most reported but hardly the only type of error in opinion research. It is treated as more important than it is and than the other types of errors because (a) it has the veneer of precision because it is a number and (b) it easier to understand and better researched than other categories of errors. Here is a simple measure of the its importance: "sampling error" so-called is taught in the introductory course but other types of errors are saved for later in a student's learning. Here's another: If you read any questionnaire carefully and think seriously about the methods used to gather the data, you almost always will find sources for potentially greater "error" than estimation error in what is reported.
6. In fact, a legitimate argument can be made that estimation errors are not really an applicable statistic for most opinion polls we see. The underlying assumption of sampling error is that the sample in question is random, and random has a very precise statistical definition. For a host of reasons, the samples in most polls do not qualify as random in a strict sense and, in too many cases, even under the loosest standards. Harris or Gallup (forgive me for mot remembering which) used to report a table of mathematical estimation error ranges but also something called ranges generated "from observation." I do not recall that they ever explained the source of the observations but found the presentation refreshing as an implicit statement about the limitations of statistical error calculations.
7. Two final observations from reading comments. As consumers and practitioners, recognize that political arguments are still political arguments even when they are dressed up with statistical language. And, finally, do not assume that any pollster is part of a larger conspiracy against your preferred candidate until you have ruled out (a) incompetence and (b) the possibility that things are not as rosy as you want them to be.
Harrison Hickman
Global Strategy Group, LLC
P.S. Not to suggest that there is a day-(or period-)of-the-week effect in the Gallup data, but as of a few minutes ago, it seems that the stop-the-presses 10-point "lead" Obama enjoyed this weekend is now four points. An up-to-date version of our original charts is below, including a line based on calculation beginning the week after Super Tuesday.
By Guest Pollster on April 1, 2008 2:53 PM
| Permalink
| Comments (1)
| TrackBacks (0)
March 28, 2008
By Guest Pollster
[Today's Guest Pollster contribution comes from Alex Lundry, research director at the Republican polling firm, TargetPoint Consulting.]
TargetPoint Consulting recently partnered with the Cook Political Report and RT Strategies, adding a new political research question called the Net Promoter Score (NPS) to their most recent national omnibus survey (March 6-9, N=802). This measure, adapted from the world of consumer research, attempts to measure voter enthusiasm and passion for a candidate. The results provide some new understanding to how the general election for President could shape up given either an Obama or Clinton candidacy.
First introduced by Frederick Reichheld in the Harvard Business Review and since popularized in his book, "The Ultimate Question," the NPS is used in the business world as a customer satisfaction metric, measuring a customer's likelihood to recommend a product, brand or company to someone else. This captures a number of difficult-to-quantify emotions, attitudes and preferences, by posing it as a recommendation. A recommendation is the ultimate endorsement, showing just how passionately you feel about a particular company or product. A recommendation means putting your own reputation on the line; an indication of loyalty, passion, and even the latent potential for word of mouth buzz.
The question is simple:
On a 0 to 10 scale, with 0 being "not at all likely" and 10 being "extremely likely," how likely is it that you would recommend voting for [INSERT CANDIDATE NAME] in the next election to a friend or colleague?
The NPS is calculated by subtracting the number of detractors (ratings of 0-6) from the number of promoters (ratings of 9 and 10). In the business world, +16 is the median score of more than 400 companies across 28 industries; CostCo has one of the highest known scores at +81. (See the NPS website for more details and similar statistics).
Studies have shown a direct and significant correlation between a business' score and company growth - specifically, a 7 point increase in overall NPS or a 2 point reduction in the percentage of detractors can each account for one percent of positive growth, thus indicating the potential electoral consequences of this measure once adapted to political polling.
To be fair, the NPS is not without it's own set of detractors and it's validity in the political world remains to be seen. For now we can only speculate about any correlation with electoral outcomes. Nonetheless, there is some promising historical data: TargetPoint began tracking the NPS on a generic congressional ballot in September of 2005 through August of 2006 and the results did seem to forebode the Republican fall from favor and the impending Democratic advances of that November. During that time the GOP NPS had a distinctively downward slope, falling from a high of +56 to a low of +11; meanwhile, the generic Democratic NPS trended upwards from a low of +32 to a high of +56.
But what about this year's election? In an Obama/McCain match-up McCain leads 45-43, but the NPS indicate some form of an "enthusiasm advantage" for Obama: among Obama voters, the NPS is +28 (53% promoters minus 25% detractors); while 48% of McCain voters are promoters and 31% detractors for a NPS of +17. Hence an Obama advantage of 11 points.
The Clinton/McCain ballot (McCain leads 47-45) again shows a Democrat enthusiasm advantage, though a slightly smaller one of 8 points (McCain: 44% promoter, 33% detractor, +11 NPS; Clinton: 48% promoter, 29% detractor, +19 NPS).
Though there is little surface difference between the candidates, deeper analysis indicates two critical demographic differences: enthusiasm among youth and Independent voters. The NPS among Independents voting for Obama (+30) is a stunning forty-nine points higher than the score among Independent McCain voters (-19). Interestingly, McCain actually wins Independents against both Clinton and Obama, but his Indy voters are much less enthusiastic than either Obama's or Clinton's. Clinton's NPS among her independent voters is also negative (-5), and a full 45 points short of Obama's. It appears that a Clinton candidacy would remove any passion or enthusiasm among Democrat-voting Independents.
Finally, we see nearly identical performance among 18-40 year olds. McCain's NPS among this age group is +7 and -5 against Obama and Clinton respectively; Clinton actually performs worse than McCain here, with a negative score of -13, while Obama dominates at +32. Again, we are left wondering what would happen to this youth enthusiasm should Clinton become the nominee.
Keep in mind that these are scores among people already voting for that particular candidate. While a vote is still all that matters on Election Day, a recommendation driven campaign can produce new votes faster, cheaper and in a more trustworthy and impactful way than traditional campaign appeals of advertising, direct mail and robo-calls.
By Guest Pollster on March 28, 2008 1:39 PM
| Permalink
| Comments (5)
| TrackBacks (0)
March 14, 2008
By Guest Pollster
(Today's Guest Pollster's contribution comes from Professors Robert S. Erikson and Karl Sigman of Columbia University.)
In late February, SurveyUSA interviewed 600 registered voters in every state for a total of 30,000 interviews, ascertaining preferences in a McCain-Obama and a McCain-Clinton race. The focus was a new set of electoral maps of red and blue states based on who led each state in the survey. Based on who won each state in the SurveyUSA survey, Obama defeats McCain 280 to 258 while Clinton defeats McCain 276 to 262 in the Electoral College.
Of course SurveyUSA's mammoth undertaking at best presents a snapshot of the states at one point in time. And even if all the niceties of polling were perfectly met, the allocation of states as "red" or "blue" is problematic due to sampling error. Here, we take the analysis of the SurveyUSA 50 state polls one step further. Rather than assign states based on who leads in the state surveys, we assign states probabilistically to the Democratic or Republican candidate based on the SurveyUSA state polls. Then, based on these probabilistic estimates, we ask the question, given the SurveyUSA results, what are odds of an Obama or Clinton victory in the Electoral College?
To do this, we conducted one million simulations (in MATLAB) of the Obama-McCain contest and then one million more simulations of the Clinton-McCain matchup. In each case we assume that the state estimates were correct except for sampling error. Using sampling theory and the assumption of simple random sampling, we draw one million estimates of the vote for each state. In each case we draw from a normal distribution with the observed mean (percent Democratic vs. percent Republican) and the standard deviation determined by the number of respondents in the state reporting a preference (always slightly under 600).
What do our results show? First, we pooled the state polls to ascertain the national vote, weighing each state's percent in proportion to the size of its House delegation. We also assign the District of Columbia as a 436th district and assign each Democratic candidate 85 percent of the vote to McCain's 15 percent. With these assumptions, the national popular "vote" is tight as of late February. Obama wins 51.5 percent versus McCain's 48.5 percent. Clinton also wins by an even razor thin margin, 50.7 to 49.3. With 30,000 cases, both estimates are statistically significant. McCain would be in the actual popular vote lead less than one time in 20.
That being said, our simulations yield a 88% chance of Obama beating McCain (with 306 Electoral College votes on average versus 233 for McCain), and a 74% chance of Hillary beating McCain (with 285 Electoral College votes on average versus 253 for McCain). About one percent of our simulated outcomes were Electoral College ties. (We ignored within-state variation in Maine and Nebraska, which divide their electoral votes by district.)
On the one hand, we find the expected numbers of electoral votes (the average from the simulations) for Obama or Clinton to be slightly higher than SurveyUSA reports. On the other hand, there is sufficient variance in the outcomes, so that McCain wins a nontrivial portion of the simulations, even with Obama as the opponent. Our two million simulations remind us that the popular vote winner is not always the Electoral College winner, although probably due mainly to chance -- the lottery aspect of the Electoral College -- and not any identifiable partisan bias in the 2008 Electoral College.
________________________________________
We thank Linda Liu for her technical assistance.
By Guest Pollster on March 14, 2008 2:18 PM
| Permalink
| Comments (40)
| TrackBacks (0)
January 15, 2008
By Guest Pollster
(Today's Guest Pollster contribution comes Professor Helmut Norpoth of Stony Brook University).
New Hampshire voters may mystify pollsters and pundits, but they have acquired an uncanny sense of picking candidates that go on to the White House. Whatever accounts for Hillary Clinton's surprising showing in her party's primary in New Hampshire, that victory makes her the best bet for Democrats to win the general election in November; likewise, John McCain's victory in the Republican primary in New Hampshire makes him the best hope for the GOP to retain the White House in November. These predictions are derived from a forecast model I developed that uses primary performance as the sole short-term predictor of the vote in the general election (the "Primary Model"). I have applied the model, with slight modifications, in the last three presidential elections, in which it correctly predicted the winners of the popular vote several months before Election Day. (See my 2004 paper in PS: Political Science & Politics). A race between the two New Hampshire winners, so the forecast, would be a nail-biter, with Clinton edging McCain by a margin of just a single percentage point of the two-party vote.
The use of primary elections to predict the outcome of the vote in the general election has some compelling advantages. One, it puts the estimation of a forecast model on a firm footing by letting us use elections all the way back to 1912, when presidential primaries were inaugurated. Two, it makes it possible to include both incumbent and opposition candidates in the model; granted, the incumbent candidate's performance may prove more powerful, but the effect of the out-party's primary showing is not negligible. And finally, the use of primaries as a predictor permits an unconditional forecast of the November vote at a very early moment. No ifs and buts. If one is willing to go with the outcome of the New Hampshire Primary, one can do it right now. The only uncertainty that remains is which of the match-ups will result from the nomination process. Chances are we may not have wait until the national conventions.
To measure primary performance in a standard format that allows for comparison across elections with varying numbers of candidates, I use an equivalent of the two-party vote in general elections. A candidate's primary showing is expressed as his or her vote relative to that of the winner (or in case of the winner in relation to the second strongest candidate). For incumbent-party candidates, the measure is adjusted, depending on whether they are sitting presidents or not. Moreover, the New Hampshire Primary is used only since 1952, when the state switched to a presidential-preference type of primary; prior to 1952, the model relies on the vote in all primaries.
Even though primary performance is the key, giving the model its name, the Primary Model also enlists a cyclical pattern of the presidential vote: the tenure of a party in the White House typically lasts between two to three terms. A compelling explanation for that dynamic is the term limit in presidential elections. Except for FDR, American presidents have eschewed running for more than two terms; and have been barred from doing so since then. The rule guarantees that incumbent presidents are missing from those contests in some periodic fashion, as is the case in 2008. In many such instances the absence of a sitting president with a high degree of popularity may improve the chances of the opposition party of capturing the White House. Given his high approval rating, Bill Clinton's ineligibility in 2000 probably hurt the Democratic prospects that year, although the absence of a much less popular George W. Bush in 2008 may be a blessing for the GOP. In any event, elections without a sitting president in the race tend to favor the opposition party more than elections with an incumbent running for another term. The Primary Model handles this dynamic by way of an autoregressive process (the presidential vote in the two previous general elections). In addition, given the use elections as far back as 1912, the model applies an adjustment for pre-1932 long-term partisanship.
From 1912 to 2004, the out-of-sample forecasts of the Primary Model pick the winner of the popular vote in 23 of the 23 elections, with 1960 being the only exception (and yes, that record includes Gore's popular vote win in 2004). The prediction equation for the presidential vote in 2008 (expressed as the Democratic share of the major-party vote) is:
.361 (RPRIM - 55.6) (-1) + .124 (DPRIM - 47.1) +.368 (VOTE04) -.383 (VOTE00) + 50.7 = .361 (RPRIM - 55.6) (-1) + .124 (DPRIM - 47.1) + 49.4
where RPRIM and DPRIM represent the primary support of the Republican (incumbent party) and Democratic (opposition party) nominees for President, capped within a 30-70 percent range, and Vote04 and Vote00 the Democratic vote shares in 2004 (48.8%) and 2000 (50.3%). The measure for the Republican candidate is inverted (-1) because the Democratic vote is used as the dependent variable. The formula produces the following forecasts of match-ups between the leading contenders in either parties (the vote for each match-up being the Democratic percentage of the two-party vote):
The PRIMARY MODEL predicts that in a race of New Hampshire Primary winners, Democrat Hillary Clinton would narrowly defeat Republican John McCain in the November general election (50.5 to 49.5 percent of the two-party vote). The predicted margin of victory, however, is so small that the confidence attached to this forecast is less than 60 percent, given the size of the forecast standard error (2.5). In match-ups between the Republican primary winner and Democratic primary losers, McCain would end up in a virtual tie with Barack Obama (49.9 to 50.1 percent) while defeating John Edwards (52.1 to 47.9 percent) by a margin close to one unit of the forecast standard error (2.6). At the same time, in match-ups between the Democratic primary winner and Republican primary losers, Clinton would dispatch Mitt Romney, Mike Huckabee, and Rudolph Giuliani by margins way beyond that error range. Finally, in match-ups between primary losers, both Obama and Edwards would beat any of the Republicans, and quite handily so in most cases.
That is no sign of partisan bias. Rather, it has to do with the Model assigning more weight to the primary performance of incumbent-party candidates than to the performance of out-party candidates. Nominating a primary loser, or even a candidate with a lackluster primary showing, costs the incumbent party more dearly than it does the out-party. Candidates not listed in the forecast table would do no better than the weakest one in their respective parties.
By Guest Pollster on January 15, 2008 1:35 PM
| Permalink
| Comments (7)
| TrackBacks (0)
January 13, 2008
By Guest Pollster
(Editor's note: Today's Guest Pollster contribution comes from Professors Robert S. Erikson of Columbia University and Christopher Wlezien of Temple University).
Does the world need one more explanation for the historic failure of the polls to predict Hillary Clinton's victory in the New Hampshire primary? We offer another possible account. Ours does not require unusual last-minute voter shifts in preference, voters lying to pollsters, or any disconnect between the campaign story line in the media and voter decision-making voters.
We suggest as the possible culprit the way pollsters' employ their likely voter screens. Pollsters may have been tricked not by voters shifting their candidate preferences but by a rapid shift in enthusiasm by Clinton supporters at the last minute. It may be that significant numbers of Clinton supporters were uninclined to vote at the time when the pollsters were doing their final interviews but then regained their interest just in time to vote. In short, the surge to Clinton could have been simply due to uncounted Clinton supporters who the pollsters dismissed as unlikely voters regaining their interest in voting.
According to most accounts, the late Clinton gains stemmed from sympathy for Hillary after her rough treatment in the media, Hillary's response to the questioning of her likeability in the final debate, and her tears on election eve. But how did this response come about? Was it due to truly undecided voters with their blank slates turning overwhelmingly to Hillary? Exit polls show no evidence of this. And it is unlikely that voters tuning in late would see the flow of the news moving in Hillary's direction. It is the idea that late-deciders could have done so that is so jarring to media watchers.
If late-deciders did not split for Hillary, maybe it was Obama supporters changing their minds? But it is even more implausible that voters who followed the campaign and settled on Obama as their choice would follow the late news and see a reason to vote for Hillary. Once people "make up their minds" in a campaign they rarely change and then only for seemingly good reasons. Did Obama supporters have reason to shift? Would the internal dialog of massive numbers of voters be: "I support Obama because he is such an exciting candidate...No wait, Hillary just shed a tear so I'll vote for her instead"?
Rather than voters deciding late for Hillary or shifting late to Hillary, we posit that her proportion of eligible voters in the New Hampshire primary was fairly steady in the final weeks. What changed was the enthusiasm of her supporters. It may be that Hillary supporters followed the news and became disillusioned by her decline in Iowa, her loss of momentum, and the general negative arc of her campaign. They were watching and they were responding to the media's storyline. Their response was not to shift to another candidate but to become dispirited. If interviewed by pollsters, their lessening enthusiasm placed them disproportionately in the "unlikely voter" column. Then, after the pollsters stopped calling, Hillary's supporters gained the enthusiasm necessary to motivate them to vote. This may be because Hillary showed her more human side late in the campaign or because it was her campaign was on the brink or for other less obvious reasons. The point is that the preferences of these voters were undercounted by pollsters. No unusual number of previously undecided voters or former Obama supporters is necessary to account for her late surge in the polls.**
Is our story true? We know that shifts in net enthusiasm from one candidate's supporters to the other's are more volatile than shifts in net preference. We also know that pollsters can be very sensitive to these shifts in enthusiasm when identifying likely voters. (See our paper from 2004 on "Likely Voters and the Assessment of Campaign Dynamics" in the Public Opinion Quarterly). Was it simply a very late shift in enthusiasm that caused the New Hampshire polls to go wrong?
Pollsters hold in their data banks the evidence that would tell if our conjecture is right or wrong. Our suspicion is that voter preferences among potential Democratic primary voters were more stable over the campaign's final weeks than generally realized. This shifting dynamic evident in the polls, we suggest, was exaggerated by daily shifts in enthusiasm that caused shifts in the composition of who gets counted as "likely voters." If likely voters first shifted against Hillary and then for, the shifting membership of the "unlikely voters" may have "surged" back and forth in the opposite way. It would be interesting to see if this was the case.
**Of course the pattern also could be explained by changes in enthusiasm among Obama supporters that mirrored what we have posited for Clinton supporter, flowing after the big victory in Iowa and then ebbing after the pollsters left the field.
Typos corrected.
By Guest Pollster on January 13, 2008 2:40 PM
| Permalink
| Comments (11)
| TrackBacks (0)
November 19, 2007
By Guest Pollster
[On November 6th, Charles Franklin posted an analysis of recent trends in public opinion on the Iraq War. In response we've recieved a Guest Pollster's contribution from political scientist Alan Abramowitz, the Alben W. Barkley Professor of Political Science at Emory University in Atlanta, Georgia.]
The claim that there has been a significant shift in public opinion toward the war is simply not supported by recent polling data. For example, a new CNN/Opinion Research Poll finds opposition to the war at an all-time high of 68 percent. The latest NBC/Wall Street Journal Poll finds that 27 percent of Americans approve of the president's handling of the war, down 3 points from September and almost identical to the levels of support from the first half of the year. This same poll finds that the war remains easily the most important issue in the minds of Americans--26 percent named the war as the most important problem for the federal government to address with health care a distant second at 16 percent. 46 percent of the respondents chose Iraq as the first or second most important problem compared with 34 percent who chose health care as the first or second most important problem. Other issues including terrorism, illegal immigration, and global warming finished far behind.
The war remains enormously unpopular and major political liability for the Republican Party. The new ABC-Washington Post Poll finds Democrats favored over Republicans on the war by a 16 point margin, slightly higher than the Democratic margin earlier this year and last year.
The claim that public opinion has shifted on the war appears to be based almost entirely on a small uptick on one measure--opinion about how the war is going. There has been a small improvement on this question, presumably in response to reports of decreasing violence and, most importantly, decreasing U.S. casualties. But this shift is not indicative of any broader shift in public opinion toward the war. Opposition to the war remains as high as ever as does support for a withdrawal timetable. And Iraq clearly remains the most salient issue in the 2008 election.
By Guest Pollster on November 19, 2007 3:31 PM
| Permalink
| Comments (3)
| TrackBacks (0)
October 18, 2007
By Guest Pollster
[This Guest Pollster's Corner contribution comes from Lydia Saad, Senior Editor of The Gallup Poll, responding to criticism posted earlier today by Alan I. Abramowitz.]
Alan, I see your point about how Gallup's question explaining the difference between Bush's income threshold and the Democrats' threshold could have confused respondents. You overlook the fact that we set up the question with this introduction to the series: "As you may know, Congress is considering a bill that would increase the number of children eligible for government subsidized health insurance, but the Democrats in Congress and President Bush disagree on how much to increase the program." But your point is well-taken.
However, our question measuring concern about the Democrats' bill being a step toward socialized medicine isn't "biased" -- it was intentionally written to convey Bush's counterargument. That was the intent -- to test the strength of socialized medicine as a counterargument. And indeed we found a slim majority willing to say they are concerned. We didn't conclude from this that Americans think the bill WILL lead to socialized medicine. As you note, we merely said that Americans are sympathetic to the argument: "Americans are also generally sympathetic to Bush's concern about the program leading to socialized medicine."
It is always a challenge to write clear and balanced questions about complex policy issues. Along those lines, I would go further than your critique of Gallup, and submit that all of the public polling on SCHIP I've seen thus far can be criticized in one regard or another. Yesterday I saw this question by CNN which seems to suggest that SCHIP is a new $35 billion program for children in middle income families, and that Bush opposes the program. I see what CNN was trying to do (isolate the question to the program expansion) but their wording just doesn't succeed at accurately describing what the veto/override conflict is all about.
26A. As you may know, President Bush vetoed a bill passed by Congress that would create a program to spend 35 billion dollars to provide health insurance to some children in middle-income families. Do you think Congress should vote to create that program by overriding Bush's veto, or do you think Congress should vote to block that program by sustaining Bush's veto?
The ABC question you applaud is another reasonable attempt, but still conveys the sense that the policy choice is between supporting the Democrats' plan and not providing any insurance coverage for "millions of low-income children." And since they ask if Congress should vote to override Bush's veto, they should have a follow-up asking what should happen if that override fails: i.e. Now that Bush has vetoed the bill, should the Democrats and Republicans in Congress work together to pass a new compromise bill, or should they let the program expire?
Otherwise we are just falling into the same political traps the Democrats and Republicans are setting for each other, and not really finding out what kind of government sponsored children's health care coverage Americans want for the country.
Americans probably don't have a great command of the details of the SCHIP debate, but half say they are paying very or somewhat close attention to it. That's about the midpoint for public attention to policy-debates in Washington. Americans clearly have some opinions worth tapping, and the challenge is to probe further for a more thorough and accurate understanding of whether Americans would rather have the existing program that covers families earning up to twice the poverty level, or whether the program should be expanded to include families earning more than that. Separately, we can find out who Americans would blame if the program expires: Bush for vetoing the congressional bill, or the Democrats for not being willing to pass a compromise bill.
None of the polling I've seen thus far -- neither on a question by question basis, nor in its totality -- answers those questions for me.
Lydia Saad
Senior Editor, The Gallup Poll
By Guest Pollster on October 18, 2007 7:46 PM
| Permalink
| Comments (10)
| TrackBacks (0)
By Guest Pollster
[Today's Guest Pollster's entry comes from Alan I. Abramowitz, the Alben W. Barkley Professor of Political Science at Emory University in Atlanta, Georgia. He has also been a frequent contributer to the blog Donkey Rising.]
The Gallup Poll has just released a report on public attitudes regarding President Bush's recent veto of a bill expanding the State Children's Health Insurance Program (SCHIP). Gallup reports that a majority of Americans trust the Democrats in Congress more than President Bush on this issue. That's consistent with what other polling organizations have found. However, Gallup reports that a majority of Americans support the President's position on where to set the income threshold for SCHIP eligibility and that "Americans are also generally sympathetic to Bush's concern about the program leading to socialized medicine." A close examination indicates however, that the questions on which these conclusions are based are clearly biased.
Here's the question concerning the income threshold:
As you may know, the Democrats want to allow a family of four earning about $62,000 to qualify for the program. President Bush wants most of the increases to go to families earning less than $41,000. Whose side do you favor?"
The problem, of course, is that the question implies that the Democrats, in contrast to President Bush, do not want most SCHIP funds to go to families earning less than $41,000. But this is not true. In fact, under the legislation passed by Congress, the large majority of SCHIP funds would go to families earning less than $41,000.
The second question is even worse:
How concerned are you that expanding this program would create an incentive for middle class Americans to drop private health insurance for a public program, which some consider to be a step toward socialized medicine? Are you very concerned, somewhat concerned, not too concerned, or not concerned at all?"
Here the respondents are presented with a totally one-sided argument using the loaded term "socialized medicine." It is actually surprising, given this wording, that a substantial proportion of the respondents indicated that they were not concerned.
Given the biased wording of this question, it is not surprising that the results are out of line with those of other recent polls on this topic based on a more balanced wording. For example, an ABC-Washington Post Poll conducted from September 27-30 of this year asked the following question:
There's a proposal to increase federal spending on children's health insurance by 35 billion dollars over the next five years. It would be funded by an increase in cigarette taxes. Supporters say this would provide insurance for millions of low-income children who are currently uninsured. Opponents say this goes too far in covering children in families that can afford health insurance on their own. Do you support or oppose this increased funding for this program?"
Seventy-two percent of the respondents in this survey favored the proposed expansion of the SCHIP program.
Update: Gallup's Lydia Saad responds
By Guest Pollster on October 18, 2007 3:07 PM
| Permalink
| Comments (3)
| TrackBacks (0)
July 20, 2007
By Guest Pollster
Brent Seaborn, Director of Strategy for the Giuliani campaign, offers the following response to Charles Franklin's recent post on the trend in support for Giuliani.
By Brent Seaborn
Director of Strategy
Rudy Giuliani Presidential Committee
Despite assertions to the contrary on this site, the Giuliani campaign is in a very strong position at this point and is clearly best-positioned to win the primary. Let me point out a few differences between the McCain and Giuliani trend line:
- When Mayor Giuliani first announced his candidacy for president, he received a considerable bounce in the polls. We anticipated that the race would close after our initial bounce -- in memos written on March 22 and June 22. I wrote, at the time, we should expect polls to tighten, as they have.
- As the race developed early in the spring, the race quickly but briefly, developed in to a two-way race, and our initial bounce extended into the beginning of this two-way race. The two-way race divided most of the Republican primary vote between 2 major candidates -- the nature of a two-way race generally forces undecided or leaning voters to make a choice between the leading candidates and many broke our way.
- As McCain's trend line declined Mitt Romney's slowly rose and Fred Thompson entered the race. Senator McCain is still a candidate for President and continues to receive a substantial vote share.
- Fred Thompson now seems to be the beneficiary of an announcement (or pre-announcement) bounce. And Fred Thompson's entry to the campaign has effectively made this now a four-way race.
- After months as the frontrunner and the addition of a fourth candidate to the GOP primary it is notable that we are in roughly the same spot we were in before our bounce and when this was still a three-way race. In a four-way (or as your graph suggests a five-way race), a trend line from the first of the year until now, excluding our "announcement" bounce, is virtually flat.
- I also note this paragraph:
"Are there any bright spots for Giuliani, other than money? Yes. There is a hint in the data that his decline may have slowed and support stabilized in the last month. In the first plot above, the blue line is my standard trend estimator which is rarely mislead by "blips" in the polls, but which is also a bit slow to be convinced that a change of trend has occurred. The red line in the plots is my more sensitive estimator-- quicker to notice a change, but also more easily fooled by "changes" that turn out to be phantoms. The red estimator has flattened out recently for Giuliani, and currently sees relative stability at about 26%. The blue estimator instead sees continued decline and a current level just under 25%. If the red estimator is right (and it often isn't) then perhaps the worst days of declining support are now behind Giuliani, at least nationally. If so, his campaign can try to get the trend moving up instead of down, but at least the decline has stopped. Unlike McCain, Giuliani has the money to try to make the numbers turn up."
I believe this is more than a blip. The red trend line will begin to pull the blue line up to meet it. In fact, if one looks at major media polls over the last month most of them show Rudy Giuliani receiving 30% or more of the GOP primary vote. In fact the mean of the major media polls over the last month is 29%.
Newsweek 6/20-6/21: RWG 27%
CNN 6/22-6/24: RWG 30%
FOX 6/26-6/27: RWG 29%
CBS 6/26-6/28: RWG 34%
USA Today/Gallup 7/6-7/8: RWG 30%
AP 7/9-7/11: RWG 21%
Gallup 7/12-7/15: RWG 33%
FOX 7/17-6/18: RWG 27%
Major media polls actually show Rudy Giuliani ahead of even the red tend line. And I would add that the red trend line, at this point in the race, disproportionately accounts for the Rasmussen Polls that by their regularity are a drag on Mayor Giuliani's trend line.
- Overall, we are very pleased with our performance in national polls. We are aware we will continue to face challenges and the race will likely continue to close. But we believe we have a real and solid base of support and we will remain competitive as this race evolves.