August 27, 2006 - September 2, 2006


A Guide to Our Charts and Data

Topics: pollster.com

While we are excited about the chart and data pages debuting today on Pollster.com, please keep in mind that the site remains under development. We are planning constant upgrades to the Polls section, and we will update this post to reflect changes and additions. For our launch, we are only posting data for the most competitive races. Within the next few days we will be adding poll data, where available, for other statewide races.

To find a chart and data for a particular race: Pull down the menu on the "Find-a-Chart" box in the upper right corner of the page. Choose a state. Then choose a race, Senate or Governor.

Each chart page will feature a table at the top displaying the candidate names and party affiliations. The first column of data will show the average percentages favoring each candidate from the last 5 public polls released for that race.** To the immediate right, the table will also display the margin separating the top two candidates. The last column will display an average of the last 10 public polls in each race.

The chart that appears below includes three elements which users can click on or off using the controls that appear just below the chart:

  • Poll results -- dots representing the percentages for each candidate
  • Polling trend -- a line for each candidate showing the trend in the last-5-poll average over time
  • Confidence intervals -- two faint lines showing the upper and lower range of the reported margin of error for each poll result.

The color of each chart element corresponds to the party affiliation of the candidate. Democrats are in blue, Republicans in red and independents, where they appear, in grey.

At Pollster.com, we intend to report the results of every public political poll that claims to project the views of the electorate, including those conducted using non-probability Internet panel samples. While the tools are not yet ready, we hope to provide readers with the ability to compare and contrast the results from newer survey methodologies.

For now, we have included the "margins of error" reported for non-probability Internet surveys, such as those conducted by Zogby International. The practice of reporting a margin of error for such surveys has been officially condemned as misleading by the American Association for Public Opinion Research (Interests declared: Mark Blumenthal serves on AAPOR Executive Council). I have written about this subject previously and will have more to say about it in the coming months.

A table appears just below each chart displaying results from all of the polls plotted in the chart. A click on any of the column headings will sort the table accordingly. Click on "pollster" for example, and it will sort on the pollster name. "Method" in the far right column indicates the methodology of the poll:

  • "Phone" indicates polls conducted using conventional telephone methodology involving a live interviewer 
  • "IVR" indicates polls conducted using an automated (or Interactive Voice Response) methodology rather than a live interviewer 
  • "Internet" indicates polls conducted over the internet, usually involving non-probability sample methods
  • "Mail" indicates a probability sample conducted by U.S. mail

For our launch, we have included a feature that allows users to remove individual polls from the chart. Click on any row of data in the table and its background color will change to grey, indicating that the chart above no longer plots its data. Users can continue to click to remove as many polls from the chart as desired. Clicking the "reset" link just above the "method" column in the top right corner of the table will restore all data to the chart.

Please note that clicking to remove data from the chart will not recalculate the averages in the header bar at the top of the page.

** Also note that the 5-poll average reported at the top of the page may differ from the averages we provide to Slate. The Slate/Pollster averages exclude results from Internet surveys conducted using non-probability samples.

Finally, if you are having any problems with the chart or would like to recommend features for future upgrades, please email us as questions@pollster.com

Introducing Pollster.com

Topics: pollster.com


Pollster.com is the new home of Mystery Pollster, the blog that has labored to demystify the art and science of political polling for the last two years, but it is also much more. Our Polls feature will take you to pages with complete listings of all the public polls available for the most competitive races for Senate and Governor with an important bonus: Interactive charts that show you how the poll results compare to each other as well as trends over time.

Before you dive into the data pages, let me tell you about the incredible team behind Pollster.com. Regular MP readers will notice a similarity between our charts and the stellar graphics produced by our friend Charles Franklin, professor of Political Science at the University of Wisconsin and creator of the blog PoliticalArithmetik. Franklin is a central part of the Pollster team and will also provide frequent commentary here on the Pollster blog as well as lead in the development of new ways to visualize results graphically.

By the way, today also marks the debut of our strategic partnership with Slate Magazine. We have worked with Slate to create an Election Scorecard that will track the daily trends in the race to control the U.S. Senate, the House of Representatives and key Governorships in 2006. With the help of Charles Franklin, I will write a daily update for Slate through Election Day on where those races stand. Links to that update will also appear here daily.

The owner and primary sponsor of Pollster.com is the survey research company Polimetrix, Inc. We have been hard at work over the last few weeks getting this site up and running, with the help of some very very talented people. Special credit belongs to the very talented development teams at Interactive Strategies, CDev Technologies and Polimetrix.

We have much planned for Pollster, although for now you will mostly see room for expansion. We plan to add contributors to the blog pages who are professional pollsters of different stripes and types. Some may post regularly. Some may contribute only once in a blue moon when they have new results to discuss or if they have a bone to pick with something we have written.

In a post below, I provide an initial guide to the charts on Pollster. But keep one thing in mind, especially if you dive into the poll data and charts right way. This site is still very much in its "beta" stage. Some tasks are unfinished; some things will not work as we had hoped in the first few days. But be patient. We will be improving and upgrading everything in the days and weeks ahead. It's going to be an exciting couple of months. And if you like what you see so far, well, hang on, because as some well-known politician used to say, "you ain't seen nothing yet." And unlike that campaign promise, you won't have to wait until after Election Day to see what we mean.

To those who know me, yes, this new gig will be more than a labor of love. While I will continue to consult for my remaining political clients this cycle, come November, my role as publisher and editor of Pollster will be my full time day job (which means no more scare quotes around those last two words).

Finally, to MP's loyal readers: The mysterypollster.com address will begin redirecting you to this page later today, so you need not change your bookmarks or blog roll links. Your continuing support for my efforts over the last two years has made this exciting new site possible. Thank you . . . and stay tuned!

The NYT Reader's Guide to Polls

Topics: Divergent Polls , Measurement

Jack Rosenthal, a former senior editor of the New York Times, filled in as the guest Public Editor" this past Sunday and devoted the column to a remarkable "Reader's Guide to Polls." The column (which also includes a kind reference to MP's coverage of the AMA's online Spring Break study) provides a helpful sampler of the various sources of imprecision in public opinion polls.  It is a worthy general primer, but as with any attempt to condense a complex subject into a very small space, a few items he covered would probably benefit from a more complete explanation. 

One example involves the reference by Andrew Kohut of the Pew Research Center to some findings from their polls on gay marriage:

The order of questions is another source of potential error. That's illustrated by questions asked by the Pew Research Center. Andrew Kohut, its president, says: "If you first ask people what they think about gay marriage, they are opposed. They vent. And if you then ask what they think about civil unions, a majority support that."

Those intrigued by that particular finding should definitely download the Pew report from November 2003 that documented the experimental findings.  Here's the key passage:

Granting some legal rights to gay couples is somewhat more acceptable than gay marriage, though most Americans (51%) oppose that idea. Public views on giving legal rights to gay and lesbian couples depend a good deal on the context in which the question is asked.  On the survey, half of respondents were asked their views on civil unions after being asked about gay marriage, and half were asked the questions in the reverse order.  When respondents have already had the opportunity to express their opposition to gay marriage on the survey, more feel comfortable with allowing some legal rights as an alternative.  But when respondents are asked about legal rights without this context, they draw a firmer line.


This context difference has little effect on core support and opposition to gay marriage itself, which is opposed by nearly two-to-one regardless of how the questions are sequenced.  But opponents of gay marriage are much more willing to accept the idea of some legal rights after they have had the opportunity to express their opposition to gay marriage. The percent favoring legal rights rises to 45% in this context, while just 37% favor the idea alone.  Put in other words, opponents of gay marriage are much more likely to accept allowing some legal rights when they have already had the opportunity to express their opposition to gay marriage itself.

Note also that the Pew surveys have shown a modest increase in support for both gay marriage and civil unions in surveys conducted since 2003.

Another topic, brought to my attention by a very alert reader, concerned this passage from the Rosenthal piece: 

Respondents also want to appear to be good citizens. When the Times/CBS News Poll asks voters if they voted in the 2004 presidential election, 73 percent say yes. Shortly after the election, however, the Census Bureau reported that only 64 percent of the eligible voters actually voted.

Ironically, as the reader points out, the Census Bureau's estimate of turnout is itself based on a survey, in this case the Current Population Study, which is prone to the same sort of measurement.  Professor Michael McDonald of George Mason University produces his own turnout statistics based on aggregate population and vote statistics.  His estimate of turnout among eligible voters in 2004 was closer to 61%, not 64%.  The Census Bureau explains the imprecision of this particular estimate in a footnote (#2):   

The estimates in this report (which may be shown in text, figures, and tables) are based on responses from a sample of the population and may differ from actual values because of sampling variability or other factors.

The "other factor" in this case is the same phenomenon that results in the over-reporting of voting in the NYT/CBS polling -- respondents wanting to be good citizens.  I suppose this example teaches that vetting survey results for publication is not as easy as we might imagine. 

Speaking of which, the Rosenthal piece also included this news:

The Times recently issued a seven-page paper on polling standards for editors and reporters. "Keeping poorly done survey research out of the paper is just as important as getting good survey research into the paper," the document said.

True enough.  But what standards will the Times now apply?  If that seven page document has been released into the public domain, I haven't seen it.  ABC News puts its Methodology and Standards guide online.  Why not America's "newspaper of record?"