One more in series of brief interviews conducted at this week's AAPOR Conference, this one with Chase Harrison, preceptor in survey research at Harvard University. Harrison describes his analysis of the accuracy of pre-election polls during the 2008 primaries.
Yet another in series of brief interviews conducted at this week's AAPOR Conference, this one with Jeff Jones, managing editor of the Gallup Poll. Jones presented findings on the interviews Gallup has conducted by cell phone with Americans living in households with non landline phone service to supplement national surveys and polls conducted in four primary states.
Starting just after 2:00 on the video, Jones discusses the impact of those additional interviews on the general election matchups between the two Democratic contenders and John McCain. While the inclusion of cell phone only households makes little difference in the Clinton-McCain contest, it benefits Obama by a net four points: Without cell phone interviews, and weighted using Gallup's usual likely voter model, McCain would get 49% to Obama's 46% (clarification: this result combines six Gallup/USAToday surveys conducted so far during 2008). With the cell-phone interviews included, the result is Obama 48%, McCain 47%.
And don't miss the cameo appearance by ABC polling director Gary Langer.
Another in series of brief interviews conducted at this week's AAPOR Conference, this one from Patrick Murray, director of the Polling Institute at Monmouth University. His paper, co-authored by Monmouth University colleague Timothy MacKinnon, concerned an experiment with random digit dial (RDD) and registration based list sampling (RBS):
More on the pollster debate over RDD vs RBS sampling here and here.
Another in series of brief interviews conducted at this week's AAPOR Conference, this one with Sarah Dutton, deputy director of surveys for CBS News. Dutton discusses findings presented at the conference (and co-authored by CBS colleagues Jennifer De Pinto and Fred Backus) regarding the effects of race and gender of interviewers on primary polling on the 2008 Democratic presidential nomination.
Dutton discusses these findings in the context of theories about the so-called Bradley-Wilder effect. I've written about that subject here and here. Politico's Daniel Libit's story today also reviews the debate over this issue during the 2008 primaries.
Another in series of brief interviews conducted at this week's AAPOR Conference, this one with outgoing AAPOR President Nancy Mathiowetz discussing her advice to pollsters and journalists on pre-election poll coverage and thoughts on pollster cooperation with the request for data from AAPOR's Special Committee on Primary Polls:
Related - Politico's Daniel Libit also filed a story today that touches on AAPOR's special committee:
In the wake of New Hampshire’s polling glitch, AAPOR convened a task force to study what had gone awry. Though its report has not been finalized, the task force’s head, University of Michigan professor Michael Traugott, says the evidence has yet to point to any “smoking gun.” [...]
AAPOR had hoped to publish the task force findings before this weekend’s convention, but Traugott says the work has been stalled by the hesitancy of pollsters to submit their methods and practices for peer review. He expects to report by mid-to-late summer, providing enough time before the general election for pollsters to tweak their methodology or improve their voter-screening questions.
“It does make for a challenge when you have public pollsters who won’t share their methods appropriately with others,” says Rob Daves, a past AAPOR president. “Science is based on transparency, and I’m not just talking about social science.”
Another in series of brief interviews conducted at this week's AAPOR Conference, this one on the topic of what to make of survey non-response rate from one of the foremost experts on the topic: Professor Robert Groves of the University of Michigan:
Another in series of brief interviews conducted at this week's AAPOR Conference, this one on the continuing growth of call phone only households with Steven Blumberg of the Division of Health Interview Statistics of the National Center for Health Statistics (NCHS):
Blumberg discusses the latest NCHS report on the continuing growth in cell-phone-only households released yesterday. The report's money quote (and chart):
Preliminary results from the July-December 2007 National Health Interview Survey (NHIS) indicate that nearly one out of every six American homes (15.8%) had only wireless telephones during the second half of 2007. In addition, more than one out of every eight American homes (13.1%) received all or almost all calls on wireless telephones despite having a landline telephone in the home.
Unfortunately, we had a bit of an audio mishap with this first interview from Frank Newport, editor-in-chief of the Gallup Poll and also AAPOR's Conference Chair for 2008. The hand-held microphone was not connected properly, so you will hear a bit more ambient sound than is optimal, but Newport's answers are clear.
Also, a verbal typo: AAPOR holds a conference, not a convention. The cause of the audio glitch (an errant adapter) slowed the pace of interviews today, but I have a few more (with better audio) that I'll be uploading shortly.
Just a quick note to say that I am in New Orleans for the 63rd Annual Conference of the American Association for Public Opinion Research (AAPOR). I will be doing video interviews and updates from the conference for the next four days. If all goes well, I will have the first posted later this afternoon, so stay tuned!
DailyKos diarist Meng Bomin has thematic maps galore on the Democratic primary results (via Sullivan):
And I am heading to catch a plane to New Orleans for the 63rd Conference of the American Association for Public Opinion Research (AAPOR), from whence I'll blog. More details later. I will be off the grid as results from West Virginia come in, but Eric will put up an open comments thread and our friend Mark Lindeman should be dropping by.
On the night before Indiana and North Carolina, [Obama chief strategist David] Axelrod appeared unusually grim and gloomy. The final night of internal polling showed Obama 12 points down in Indiana against Clinton--a disastrous collapse after two or three days of closing the gap. The campaign's pollsters cautioned that the last night's sample seemed weird and they should rely instead on the three-day rolling average of 2 points. But Axelrod feared the worst, that Wright had sunk the campaign in Indiana and possibly in North Carolina, too.
The next day, after visiting some polling stations, Obama arrived back at his hotel and stopped by the coffee shop, where he urged some curious bystanders to vote for him. When a NEWSWEEK reporter asked him about Axelrod's gloomy prognosis, Obama shrugged and said: "It is what it is. We've had a month, two months of bad stuff. It's been hard to change the storyline." He smiled and walked out to get ready for his now traditional Election Day game of basketball. If he was at all worried, as his senior staff was, he hid his concerns successfully from the outside world.
Smith concludes that this story is a "great anecdote reminding journalists -- and campaign staffers -- to be careful when looking at polls, especially the daily tracking polls." True, but the bigger caution ought to be about putting too much faith in any survey result -- and that includes quite a few in the public domain -- based on just a single night's calling.
Also, for what it's worth, Mark Halperin has a video interview with Axelrod that includes words of praise for the campaign's four pollsters.
Why have you stopped including summary numbers in your graphs?
As most of you know, our charts usually include a regression trend line (a line drawn through the points) as well as a legendd at the top of each chart that includes the most recent value of the trend estimate for each candidate. This example below shows the final candidate estimates for the Indiana Primary:
However, as Dave and other alert readers have noticed, the charts for West Virginia and all of the remaining primary and caucus states except Oregon no longer display any sort of "summary number." Their legends look like this:
The reason is that we do not plot a regression trend line when we have less than eight polls. Why not? Here is Professor Franklin's explanation posted when we started running these charts last year:
Ideally, the trend estimator should have a dozen or more polls to work with before we take the trend very seriously. When the number of polls drops too low, the trend estimator will jump around considerably if new polls are very far from previous polling and may produce jagged trend lines that are likely to change with more data. Despite this danger, we've estimated the trend with as few as eight polls, rather than stick to the safer minimum of twelve. Too many states are between 8 and 12 polls to ignore, and while we are cautious most of the trends look pretty reasonable even with less than 12 polls. When the polls are consistent with each other, the trend estimate will still be pretty good even with eight. But when there is substantial disagreement among the polls, we will get jagged or otherwise "bad" trends. Since the plots show the actual polls you can look at the data yourself and decide whether the trend is a reasonable fit to the data, or if it is erratic enough to be discounted. You decide.
Until this past weekend, the legend on charts with less than eight polls had displayed the candidate numbers for the median poll (or the average of the two middle polls when we had 2, 4 or 6 polls available. While that approach made sense to us last year, we found two practical problems: (a) the median poll was sometimes very outdated and (b) nearly everyone was confused by this approach. Many concluded the numbers in the box were clearly "wrong" as they didn't seem to match the dots on the chart. So rather than adding a confusing explanation in small print, we thought it better to err on the side of caution and simply drop the "summary numbers" when we have fewer than eight polls.
"Rasmussen Reports believes the race is over and that Barack Obama will be the nominee of the Democratic Party. We will stop tracking the Democratic race in the near future to focus exclusively on the Obama-McCain match-up."