Pollster.com

October 16, 2009

Greetings from LSU

By Mark Blumenthal

Here's a quick apology for my lack of blogging today and (in advance) for tomorrow too. I was traveling for much of today to Baton Rouge, Louisiana where tomorrow I'm participating all day in the John Breaux Symposium at the Reilly Center for Media & Public Affairs at Louisiana State University.

If you happen to be in Baton Rouge and have some free time tomorrow, tomorrow's discussion should be terrific. The topic is "Redefining Public Opinion Polling in an Age of Segmented Markets and Personalized Communication." In addition to yours truly, the panelists include our own Charles Franklin, Charlie Cook of the Cook Political Report, Scott Keeter of the Pew Research Center, Anna Greenberg of Greenberg Quinlan Rosner and Susan Herbst of Georgia Tech.

Meanwhile, two quick "outliers":

Louis Jacobson of Politfact.com did a fact check on Glenn Beck's citation of a result from the IBD/TIPP poll of physicians that I discussed on Pollster and in a column last month. Jacobson's piece includes considerable new reporting on the issue -- including the full text of the questions asked in the survey. It's worth a click. 

Finally, we overlooked a new Rasmussen poll in New Jersey today. I just added the trial heat question to our chart; we should have the usual poll update post in the morning.

By Mark Blumenthal on October 16, 2009 12:21 AM | Permalink | Comments (0) | TrackBacks (0)

October 6, 2009

Disclosing Our Own Screwup

By Mark Blumenthal

In the spirit of transparency, we need to provide full disclosure of a mistake we made and an apology to our readers and to Rasmussen Reports.

In the year since the election, we have worked to create a new collection of charts that track all available surveys not just for election trial heat results but also a series of national measures, including presidential job approval ratings, favorable ratings, the national "generic" congressional ballot, the classic "right direction wrong track" question and a handful of measures of perceptions of the economy.

We started putting up new charts after the 2008 election knowing that some would "work" -- we would find enough reasonably comparable data from a variety of sources to make for a robust trend line -- and some would not. It was also probably inevitable that we would make a mistake or two along the way.

Well, as I discovered this past week, we did. For a handful of charts, we have been republishing some extraneous data from behind the gated subscriber pages on RasmussenReports.com. The affected charts are two that track perceptions of the economy (excellent/good/fair/poor and getting better/getting worse), our Obama favorable rating chart and the three charts that track the Obama job rating by party (Democrat, Republican and independent). Rasmussen does provide some results from these questions (usually just from one answer category) on their free-to-all "By The Numbers" page. For the economic charts, that represent the bulk of the data we misused, we were filling in results from the subscriber tabs for data omitted on the By-The-Numbers page.   Compounding the error, as noted last week, in one instance we were including numbers on our Obama favorable rating chart that were actually mislabeled job approval rating results.

I could tell a long story about a small error that cascaded, but it all boils down to a lack of clear communication by me. As such I deserve and will take full blame. So there is no confusion in the future, our policy henceforth is iron-clad: We will not republish a single number in our charts unless it has already been published or released into the public domain by the pollster or sponsor.

Knowing that Josh Tucker has raised some good questions about the whole notion of gated, subscriber-only crosstabs, I want to make clear that no one at Rasmussen complained to us about this issue. We discovered it ourselves and subsequently reached out to apologize, an apology I repeat publicly today. Except for the erroneously labeled data which we have already taken down, Scott Rasmussen has kindly granted us permission to leave the remaining data in place.

In correcting our error, however, it is now clear that two of our charts -- those tracking current and retrospective assessments of the economy -- will no longer "work" as intended. Virtually all of the data going forward would be coming from the Gallup Daily tracking and, as such, our chart would add no real value to those that Gallup publishes itself (here and here). We may rework our charts using only monthly values at some point in the future, but if we do, those charts will be based on monthly releases from other organizations, and from Gallup or Rasmussen should they ever opt to put monthly summaries into the public domain.

By Mark Blumenthal on October 6, 2009 3:52 PM | Permalink | Comments (1) | TrackBacks (0)

September 17, 2009

What Happened to the PoliticsHome Box?

By Mark Blumenthal

You may have noticed that the PoliticsHome box -- the one that included links to the top news stories of the day -- has disappeared from the site. Fear not, PoliticsHome fans, the box on our site is just on a temporary hiatus. PoliticsHome US, which is run by a different organization, recently launched a redesigned site that had the unfortunate side effect of "breaking" the sidebar box on Pollster (it was stuck on September 7).

The box should return soon. Meanwhile, those who have grown to enjoy their collection of "top stories right now" and the day's "must reads" can go directly to the Politics Home site.

And if you have come to use and depend on the PoliticsHome box on Pollster.com, we would appreciate it if you would email us or leave a comment. The more readers we hear from, the sooner I can get the box repaired and back in place.

By Mark Blumenthal on September 17, 2009 12:08 PM | Permalink | Comments (0) | TrackBacks (0)

September 4, 2009

Happy Birthday to Us

By Mark Blumenthal

Pollster.com quietly turned three-years-old this week. We launched with this post and a less pretty set of charts on September 1, 2006. Since that time, according to Sitemeter, we've served up over 80 million page views during over 28 million visits. We were honored to win the Warren J. Mitofsky Innovators Award from the American Association for Public Opinion Research (AAPOR) two years ago, receive praise for "excellent reporting" of pre-election polls in 2008 from statistical visualization guru Edward Tufte and, just last month, be named one of the 50 Best Websites of 2009 by Time.com (along with the likes of Google, Facebook and Twitter). Though exhausting at times, it has been a truly rewarding adventure, and we look forward to celebrating many more birthdays in the years ahead.

But as we pause and reflect on the last three years, I want to take a moment thank those who have helped make Pollster.com a reality: Doug Rivers, who originally conceived of Pollster.com and continues to provide financial and technical support through our principal sponsor YouGov/Polimetrix; our partners at the National Journal Group: Charles Franklin who has been a valued partner in this effort from day one; our growing list of contributors; the many talented individuals who helped develop our website, database and charts (though I'll single out Jeff Lewis, Seth Hill, Ben Schaffer and Quentin Fountain for their extraordinary contributions); and finally, Eric Dienstfrey and his successor Emily Swanson, the true heros who worked the hardest to bring you an accurate and up-to-date Pollster.com every day.

And, of course, we owe the biggest thank you to all of you who visit, read and link regularly. We would not be here but for your support.

Coincidentally, Will Urquhart at at SumOfChange.com just posted a well-produced video of the complete Netroots Nation panel that Charles Franklin and I participated in last month along with DailyKos contributing editor Greg Dworkin (DemFromCT), Charlie Cook of The Cook Political Report, and Nate Silver of FiveThirtyEight.com.

If you have time to watch just one presentation, I highly recommend the one by Charles Franklin that begins at about 19:55. Among other things, Charles provides the best review I've seen yet of the philosophy that guides the way we construct our charts and analyze polling data at Pollster.com.   

My presentation begins at about 52:00 and is the made-for-TV-movie version (if you will) of the three-part-series I posted last month entitled, "Can I Trust This Poll."

By Mark Blumenthal on September 4, 2009 4:51 PM | Permalink | Comments (2) | TrackBacks (0)

August 24, 2009

Thank You, Time!

By Mark Blumenthal

Some very good news: Time has named Pollster.com to its list of the 50 Best Websites of 2009! What makes this honor especially huge is that Time's list is not limited to political or blog sites but rather features a much broader range of sites that "make your online life more efficient -- or just more fun." This year's list includes names like Google, Facebook, Twitter, Flickr, Skype, YouTube, Amazon, and Wikipedia, so we are in truly amazing company.

We are also very gratified that Time specifically recognized interactive charting features that we have worked so hard on:

Pollster also aggregates [polling] data, but it has a Web interface that allows you to remix it on the fly. Is there a poll you don't trust? Throw it out! Want a different smoothing algorithm? Change it! How much difference does it even make? Magnify the X and Y axes with a mouse-click and find out.

Thank you, Time!

By Mark Blumenthal on August 24, 2009 10:28 AM | Permalink | Comments (2) | TrackBacks (0)

Not-Quite-Post-Vacation Housekeeping

By Mark Blumenthal

Now that I'm back from a week's vacation, Emily is taking two well earned days off today and tomorrow and I will be filling in adding new polls to our charts and posting poll updates. As such, those updates will be a bit slower than usual for the next 48 hours. Apologies in advance for that.

By Mark Blumenthal on August 24, 2009 9:55 AM | Permalink | Comments (0) | TrackBacks (0)

June 30, 2009

Obama Job Approval - New "All Adults" Chart

By Mark Blumenthal

Another piece of housekeeping and one of Eric Dienstfrey's final contributions to Pollster.com. We have produced a new chart that includes only polls that report the Obama job rating among all adults. The original Obama job rating chart that includes all surveys remains in place; this new chart adds a new way of tracking the trends.

We have discussed some of the challenges posed to our charts on measures like the Obama job performance rating by pollsters whose results show big "house effects" (consistent differences when compared to other pollsters). Our philosophy has always been to try to include all polls that claim to produce representative samples -- even those based on more controversial methods such as automated polls or those that survey respondents over the internet using pre-recruited panels -- to make it possible to use our interactive chart features to compare and contrast different surveys.

The problem is that if big house effects occur, the trend lines can sometimes display phantom trends when polls with consistently different results are more frequent. This issue crops up most often in the "nose" of the trend line, which moves around more than the rest of the line as we add new polls to our database. The Rasmussen Reports surveys appear to be a big problem in this respect, mostly because they are far more numerous. However, if you use your mouse to click on Obama job ratings that tend to be higher or lower than other polls, you will also see pollsters with similar house effects that poll less often.

We offer the new all-adult-sample-only charts as one means of reducing the potential for "phantom" trends, though we have other potential improvements in the works. Please let us know what you think.

PS:  A week or so ago we also broke out party identification in two: one is based on results among all adults, one among surveys of registered or likely voters.

By Mark Blumenthal on June 30, 2009 4:02 PM | Permalink | Comments (0) | TrackBacks (0)

Welcome Emily - Farewell Eric

By Mark Blumenthal

Regular readers have probably noticed a new name appearing on the "poll update" entries on Pollster.com. Emily Swanson, a recent graduate of the University of Wisconsin-Madison, has joined the Pollster.com team and will be posting and updating our charts and tables regularly from here on out. Welcome Emily!

Unfortunately, Emily's appearance means that we are saying farewell to Eric Dienstfrey after nearly three years of relentless hard work and dedicated service. As announced a few months ago, Eric has been accepted to the Graduate Program in Film Studies at, coincidentally, the University of Wisconsin-Madison's Department of Communication Arts. So he is moving on to bigger and better things.

Sadly, today is officially Eric's last day at Pollster.com. I exaggerate not one bit when I say that the site as you know it would not exist but for his skill and tenacity. We will miss him, but wish him the best of luck in all of his future endeavors.

By Mark Blumenthal on June 30, 2009 12:03 PM | Permalink | Comments (0) | TrackBacks (0)

June 23, 2009

Alec Gallup

By Mark Blumenthal

Sad news from Gallup editor-in-chief Frank Newport:

Alec Gallup, one of the polling world's most committed practitioners and dedicated supporters of the value of polling and all around good guys passed away last night. Alec was one of two sons of Dr. George Gallup and was the long time Chairman of the Gallup Poll. Alec lived in Princeton, New Jersey. Anyone who has worked at or with the Gallup Organization over the years and who came into contact with Alec recognized what a truly unique individual he was. He literally devoted all of his life to polling -- spanning his childhood days when he worked with his father as poll "ballots" came in via train to be tabulated at Gallup headquarters up to as recently as a week or two ago, when, even in declining health, he would call up and make suggestions about what poll questions Gallup should be asking in the current political environment. Polling has never had a greater champion, and those who knew Alec personally have never had a greater friend. Everyone who knew Alec will miss him immensely.

Alec Gallup was interviewed about his father and the early days of polling nine years ago for a PBS documentary. You can read a transcript here (via Mike Mokrzycki).

By Mark Blumenthal on June 23, 2009 1:16 PM | Permalink | Comments (0) | TrackBacks (0)

June 3, 2009

Soltis on the GOP and Young Voters

By Mark Blumenthal

Kristen Soltis, a regular contributor here at Pollster.com, summarized her views on how the Republican Party can win back younger voters for the Huffington Post. Her bottom line:

In order to begin that effort, the GOP needs to have a positive message and vision that focuses on outcomes that matter to young voters. Right now, a lot of what Republicans are talking about is "less taxes" and "smaller government." But young voters are less convinced than older generations that the government tends to be inefficient and wasteful.

Among other issues, she also confronts the lack of diversity that was the subject of a widely read summary from Gallup this week that showing that 89% of Republican identifiers are white (or more specitically, non-Hispanic white) and 63% are white conservatives. Soltis:

Longer term, the Republican Party has to confront the issue of diversity. If the Republican Party retains a brand as the party tailor-made for conservative older white males, it will not survive for long. Consider the fact that younger voters represent a more ethnically diverse cohort than other generations. The issue of winning the youth vote is more and more inextricably linked to winning support among Hispanics and African-Americans.

There's much more, and it's worth clicking through for a full read.

By Mark Blumenthal on June 3, 2009 11:03 AM | Permalink | Comments (1) | TrackBacks (0)

June 2, 2009

Sometimes the Magic Works...

By Mark Blumenthal

"Sometimes the magic works," said Chief Daniel George in the 1970 classic flim Little Big Man, "and sometimes it doesn't."   The same can be said about the loess regression trend lines we plot in our charts.

When we plot pre-election poll results from various pollsters on the same charts, the trend lines usually have the helpful characteristic of minimizing the impact of outlier results and pollsters with consistent "house effects" on the overall estimate. In other words, if one of five or ten pollsters produces a consistently different result, their results do not typically skew the overall average significantly so long as the timing of the various polls is more or less random.

But for some of the national measures we have been plotting recently -- especially Obama's job and favorable ratings and the question about whether Americans perceive things to be "headed in the right direction" or "off on the wrong track" -- a few pollsters that do daily or weekly tracking are producing results with large house effects. Unfortunately that combination, along with the more sporadic timing of other national surveys, is producing the appearance of trends on some charts that are not really trends.

Last night, for example, Andrew Sullivan linked to two charts that appear to show trends in recent weeks: An uptick in the unfavorable rating for Obama and an increase in the percentage saying that things are off on the wrong track. In both cases, unfortunately, the apparent trends are an artifact of timing and house effects.

Let me explain, starting with the right direction/wrong track chart, that follows. (I am using screen shots rather than our live-embedded version here to preserve the look of the chart at the time of this writing -- follow the link to the live chart to use the filter tools yourself):

What Sullivan noticed was the recent uptick in the red line (wrong track) and downturn in the black line (right direction) at the far right (or "nose") of the trend. Now look what happens when we use our filter tool to remove from the trend the two pollsters -- Rasmussen Reports and DailyKos/Research2000 -- whose weekly tracking results provide nearly half (41 of 96) polls plotted in this chart so far during 2009. The recent trend disappears producing an essentially flat line since mid-April:

So removing just two pollsters -- and particularly the two that contributed all four of the poll released in the last two weeks -- eliminates the apparent trend. One problem we have is that these two pollsters release weekly tracks, while the others poll more sporadically. Worse, virtually all of the national pollsters released surveys just before the Obama administration reached its 100th day in office, and we have experienced something of a poll drought since.

But wait. Perhaps those two weekly tracks are catching a more recent trend that we might miss if we rely (for the moment) on the other national tracking surveys that have not produced more surveys in the last few weeks.

To check, let's use the filter tool to select only the surveys from Rasmussen and DailyKos/Research 2000. And just to be safe, I will also turn up the smoothing setting to be especially sensitive to any recent trend:

The trend is almost exactly the same as the version with these pollsters removed, but you can also see that the gap between wrong track and right direction is larger on the second chart of just Rasmussen and Research 2000 (11 points) than on the previous chart excluding those two (4 points), with virtually all of the "house effect" coming from the Rasmussen survey.

So when we look at only the weekly trackers or only the other polls separately, we see flat lines over the last few weeks. When we put them together, we see a recent upward movement on "wrong track." Why? Because when combined the weekly trackers are driving the "nose" of the trend line and the trackers -- especially the Rasmussen track -- is producing consistently different results. So as the Rasmussen results have more influence in the trend line, they tend to drive the red line up and the black line down.

Now let's repeat the exercise with the Obama favorable rating. First, the standard chart showing all surveys. The recent apparent trend is the sharp upward movement on the red "unfavorable" line:

In this case, the Rasmussen and Daily Kos/Research2000 results are six of the seven surveys conducted in the month of May (the new Gallup result was added this morning, after Sullivan's initial post). If we use our filter tool to remove the weekly trackers, the apparent recent change smooths out, reflecting the more gradual increase in Obama's unfavorable rating since the inauguration:

Again, are the trackers picking up a more recent trend that the other national surveys are missing? Here is what the chart looks like if we include only the Rasmussen and DailyKos/Research2000 polls. Here, we see virtually no trend since late March:

The last chart above also clearly shows the enormous house effect separating (in this case) Rasmussen and DailyKos/Research 2000 surveys, with Rasmussen producing consistently lower favorable and higher unfavorable ratings for Obama.

We have discussed the "why" of house effects, especially the consistent differences in the Rasmussen tracking, in previous posts. This case involves something a little more troubling for us: The way house effects and timing have combined to produce misleading "trends" that are more artifact than real. That is something we need to address in a systematic way.

Update: At the suggestion of a reader, Andrew Sullivan removed

only the Rasmussen surveys with similar results to what I obtained above.   

By Mark Blumenthal on June 2, 2009 1:18 PM | Permalink | Comments (8) | TrackBacks (0)

May 29, 2009

About That New Banner

By Mark Blumenthal

Hopefully, you have already noticed something a little different about our site today. As of this morning, a National Journal banner now sits atop our site, signifying a newly expanded partnership with the National Journal Group, publishers of the National Journal, CongressDaily, The Hotline, The Almanac of American Politics, and NationalJournal.com. While we have partnered since January 2008 -- most visibly through my weekly column on NationalJournal.com -- this new arrangement involves an even closer business relationship.

Some of you may have experienced a few bumps last night as we made the changes, but everything should be working now. If you are experiencing any unusual problems with the site, please let us know.

And as long as we are doing a bit of housekeeping, I also want to take this chance to ask for your input on both what you like about pollster and about the things you dislike or wish we would improve. I have a long list of things I would like to fix, but would appreciate your "qualitative" guidance in setting priorities. So if you can, please take a moment to leave a comment below or email me with your thoughts about the things you would most like to see upgraded or improved.

And thank you all for your continuing support!

By Mark Blumenthal on May 29, 2009 1:45 PM | Permalink | Comments (1) | TrackBacks (0)

April 24, 2009

Job Opening at Pollster.com

By Mark Blumenthal

This is something of a bittersweet post. Eric Dienstfrey, my relentlessly hard working number two here at Pollster.com, will be moving on to bigger and better things in the fall. He has been accepted into the Graduate Program in Film Studies at the University of Wisconsin-Madison's Department of Communication Arts. Congratulations Eric!

This news means that we have a job opening and big shoes to fill. This is a full-time, entry-level position in Washington DC with health care benefits, and we anticipate hiring in mid to late June. Applicants should have excellent proofreading skills, strong attention to detail and an abiding interest in political polling. While not required, the ideal applicant would also bring some previous knowledge of or experience in web site development/administration (especially with Movable Type), statistical analysis (especially with the R programming language) or database development (especially with PythonSQL).

If you are interested and would like more details on this unique opportunity, please email me and attach a resume.

Update:  We have filled the opening.  Many thanks to all that applied. 

By Mark Blumenthal on April 24, 2009 12:15 PM | Permalink | Comments (0) | TrackBacks (0)

March 25, 2009

Spring Break Housekeeping

By Mark Blumenthal

A quick update and apology for the slow pace of analysis posts over the last few days. This week is spring break for my children, and I have been trying to combine light blogging with a family visit (and rediscovering the challenge of finding uninterrupted time in the company of precocious 4 and 6-year-olds). I'll be back to full speed next week.

By Mark Blumenthal on March 25, 2009 5:17 PM | Permalink | Comments (0)

January 29, 2009

About those Dubious Polling Awards

By Mark Blumenthal

Let me start with the "my bad" portion of this entry: Three weeks ago, our colleague David Moore sent me an early draft of the "Dubious Polls Awards" commentary he co-authored with George Bishop. Moore asked for my comments, but in an oversight that speaks to my own poor management of an overflowing email inbox, I set the message aside without reading the attachment and soon forgot about it. He ultimately posted a summary here earlier today, with the more detailed version posted on stinkyjournalism.org. Had I read the draft, I would have given David feedback consistent with what follows. I apologize to David and our readers for that oversight, but I want to take this opportunity to air the issue publicly and allow readers to react and comment.   

I make no apologies, on the other hand, for giving David Moore (and by extension, George Bishop), the opportunity to blog here at Pollster. As I noted earlier this week, David brings to this endeavor a long career in the field of survey research, as an author, an academic and a former managing editor of the Gallup Poll. George Bishop is one of the most respected academic survey researchers, and though his perspective is sometimes at odds with others in the field, his work is something any serious pollster should know (particularly his book, The Illusion of Public Opinion ). If Moore and Bishop are willing to to act as provocateurs and criticize the most respected voices in the field, fine. They have the expertise to do so with authority, and constructive criticism has always been part of our mission.

I also believe that blogging works best when edited least. Holding back posts for review and revision kills the spontaneity and give-and-take that make blogging work. As Andrew Sullivan has written, readers are his best editors. "E-mail seemed to unleash their inner beast. They were more brutal than any editor, more persnickety than any copy editor."

The only rule I have tried to set for contributors here at Pollster is to follow my tone: Avoid name calling and gratuitous snark and, above all, be fair.

My problem with the Dubious Polls summary -- and the feedback I should have given Moore and Bishop -- is that it offers far too much snark and name calling with just a smattering of the smart context that they are well equipped to provide (and do a better job providing in the longer version on stinkyjournalism.org). It is also, in places, less than fair.

Consider, for example, their "top award, earning five crossed fingers:"

[It goes to] all the major media polls for their prediction of Giuliani as the early Republican frontrunner. Collectively this group, beginning more than one year prior to the first statewide electoral contest in Iowa, relentlessly, and without regard for any semblance of political reality, portrayed Rudy Giuliani as the dominant Republican candidate in a fictitious national primary.

It is certainly true that most public pollsters reported results showing Giuliani leading, consistently, throughout most of 2007, on questions that asked Republican identifiers nationally to state their preference for the Republican nomination. And it is also true that far too many journalists and pundits (and some pollsters) looked at these early results, showing Giuliani with the support of just 30% of Republicans nationally, and wrongly assumed or predicted that the former New York mayor had some sort of lock on the Republican nomination.

If Moore and Bishop had argued in their summary that we should have paid more attention to polls in New Hampshire and Iowa than nationally,or that pollsters should have done more to caution poll consumers against reading too much into those early Giuliani leads, I would agree (and did, here and here, back in August 2007). I also agree that any "predictions" of a Giuliani triumph based on those 2007 horse race polls alone ignored many political realities, including the fact that we do not hold a single-day, national presidential primary.

But is it fair to characterize as a "prediction" every horse race result released by the ten organizations Moore and Bishop list? Is it fair to use the phrase "crossed fingers" -- words that imply deliberate dishonesty -- to depict the release of those results? It feels unfair to me.

Obviously, this site is not my exclusive domain. Our goal for Pollster.com is to present a wide variety of poll and survey related content from many different authors, and we do not expect every front page contributor to agree or speak with one voice. On the question of tone, however, I want to hear from you. Please read over the Dubious Polls piece here, and the longer version on stinkyjournalism.org. What advice would you offer -- to David Moore or to me -- for future contributions?

Please feel free to comment below or email me directly. I will try to post excerpts from email in a future post (please stipulate if you prefer that your comments to remain totally off the record).

By Mark Blumenthal on January 29, 2009 2:44 PM | Permalink | Comments (2) | TrackBacks (0)

January 23, 2009

More on the New Charts

By Mark Blumenthal

As implied by the post earlier today on our New York charts, we are slowly working through the process of adding to our site all sorts of new poll charts and tables at the state level. In addition to New York, we also put up charts for two states -- New Jersey and Virginia -- with races for Governor in 2009 and one, Ohio, with a newly open Senate race in 2010. We will be adding many more, albeit gradually, in the coming weeks.

We are limited, of course, by the availability of public poll data. Some states are polled more often than others. In other states (like Virginia), pollsters have held off testing general election match-ups or will do so until contested primaries are resolved. Our aim is to post all available horse race results for all contests in 2009 and 2010 for Senate and Governor and, ultimately, for the U.S. House of Representatives as they become available.

One new feature we hope to keep consistent across states is to include tracking charts for the favorable and job ratings of each state's governor and two senators, as well as the statewide job approval rating of President Obama.

To help you find charts, Pollster.com features comprehensive index pages that list all charts and tables for all states. Each index page has a consistent URL that uses the state's two-letter postal abbreviation (e.g. www.pollster.com/polls/nj/). To make it easier to navigate to those state index pages, we will be adding two tools later today:

• The pull-down menu for "The Polls" on our masthead will have a choice labeled "Find All Polls" that will take you to the page displaying an all grey map. Clicking on a state will take you to that state's index page.

• We will also modify the text links in the sidebar box that appears at the top of our right column throughout the site to that it includes links to the index pages for all 50 states, DC and the national index page.

By Mark Blumenthal on January 23, 2009 3:41 PM | Permalink | Comments (0) | TrackBacks (0)

January 12, 2009

New Charts!

By Mark Blumenthal

If you have already glanced at our front page today, you know that we have introduced three new charts using the same Flash software that displayed pre-election polling results last year. All three are based on results from national surveys and are accompanied by tables that include links to the underlying source data:

• Barack Obama's favorable rating - displays results back as far as January 2007, when most national polling organizations first started asking Americans to rate him.

• Obama's job approval rating - currently based on questions about how Obama is handling "his presidential transition," this chart will evolve into one that tracks questions about how he handles his "job as president" once pollsters switch to that language after inauguration day.

• Right direction, wrong track - tracks answers to the question, "do you think things in this country are generally going in the right direction or are they seriously off on the wrong track," as asked by a dozen or so pollsters since Labor Day, 2008. Later this year, we hope to add more data going back further in time

These three are just the beginning. We are also planning to add many more national measures over the course of the next few months, and of course, for election tracking graphs for 2009 and 2010 races as data becomes available. Our menus and sidebar links should update within the next few days to allow easier navigation to the new charts.   

Again, the charts use the same Flash display software that we used during the fall campaign (static non-Flash graphic versions are displayed for those without a Flash capable browser). Pointing your mouse to any individual data point on the chart will pop-up information about that poll (pollster, survey dates, sample size, etc.). Clicking on that point will connect-the-dots to other results from the same organization. Options accessed through the tools menu allow you to filter out polls by any organization or by the mode of the survey, vary the sensitivity of the trend line, change the axis ranges and embed your chart, customized as you prefer, on your own blog or web page. We produced a video back in September that demonstrates most of these features.

One thing you will notice immediately is that some of these charts show more distinct "house effects" than the horse race results we typically plot. The favorable rating in particular shows big differences, owing to the sometimes very different ways that pollsters ask Americans to rate their general impressions of political leaders. Notice, for example, the way the Rasmussen surveys produce a greater unfavorable percentage for Obama and the way the CBS/New York Times wording produces lower percentages for both the favorable and unfavorable categories. I wrote about some of these differences, particularly as they affect the CBS/New York Times results, in a column back in July, along with a sidebar post that included the text of the favorable rating question asked by each national pollster.

By playing with the "filter" feature in the charts, you can get a sense for the degree to which removing any pollster or combinations of pollsters affect our overall estimate. What you will find is that the loess regression line is mostly resistant to minor "house effects," even major ones. Remove the frequently updating Rasmussen automated tracking, for example, and the overall estimate changes from 71.5%-17.8% (favorable-unfavorable) with all polls included to 72.5%-16.0% without.

By Mark Blumenthal on January 12, 2009 1:47 PM | Permalink | Comments (0) | TrackBacks (0)

January 9, 2009

Bowers Vs. 538 Vs. Pollster

By Mark Blumenthal

Chris Bowers posted a two-part series this week that compares the final estimate accuracy of his simple poll averaging ("simple mean of all non-campaign funded, telephone polls that were conducted entirely within the final eight days of a campaign") to the final pre-election estimates provided by this site and Fivethirtyeight.com.

Chris crunches the error on the margin in a variety of different ways, but the bottom line is very little difference among the methods. These are his conclusions:

• 538 and Pollster.com even, I'm further back: Pollster was equal to 538 when all campaigns are included (the "1 or more" line) and with all campaigns except the outliers (the "2 or more" line). Kind of funny that not adjusting any of the polls, and adjusting all of the polls, results in the same rate of error. To no one's surprise, my method was much better among more highly polled campaigns, but still about 10% behind the other two once poll averaging (2 polls or more) comes into play. I make no pretense about my method needing polls in order to work.

• Anti-conventional wisdom : 538 had the edge among higher-polled campaigns, which means Pollster.com was superior among lower-polled campaigns. This goes against conventional wisdom. Many thought Silver's demographic regression gave him an edge among less-polled campaigns, but that Pollster's method only worked well in heavily polled environments. Turns out the opposite was true, and I'm not sure why. Maybe Silver's demographic regressions don't work, but his poll weighting does. Or something.

• Still very close : While I was a little behind, the difference between the methods is minimal. I'm a little disappointed, but clearly anyone can come very close to both 538 and Pollster.com in terms of prediction accuracy with virtually no effort. Just add up the polls and average them. It is about 90% as good as the best methods around, and anyone can do it.

You can see the full post for details, but his calculations are in line with what we found in our own quick (and as yet unblogged) look at the same data. We simply saw no meaningful differences when comparing the final, state-level estimates on Pollster to Fivethirtyeight.

Keep in mind that we designed our estimates, derived from the trend lines plotted on our charts, to provide the best possible representation of the underlying poll data -- nothing more and nothing less. So the accuracy of our estimates tells us that the poll data alone, once aggregated at the end of the campaign, provided remarkably accurate predictions of state-level election outcomes. The fact that the more complex models used at FiveThirtyEight were equally accurate raises the question: In terms of predictive accuracy, what value did Fivethirtyeight's extra steps (weighting by past polls performance and the various adjustments based on other data and regression models) provide?

By Mark Blumenthal on January 9, 2009 4:59 PM | Permalink | Comments (3) | TrackBacks (0)

December 25, 2008

A Merry Christmas and Happy Holidays to All...

By Mark Blumenthal

Over these last two years, we are thankful to have had the opportunity to share the most exciting, compelling election of our lifetimes with you from this unique vantage point.  The last year in particular has been long and sometimes grueling, so my family and I are looking forward to taking the next week or so off.  Eric will be checking in from time to time, but I will be pretty much off the radar until the new year.

It will be a new year full of new polls and new challenges, and we are looking to bringing you new charts and analysis to follow it all.

Until then, from all of us here at Pollster.com, we wish you a Merry Christmas, a Happy Hanukkah or joy in whatever way you celebrate the holiday season.  And if I don't get the chance to say it online, a happy New Year too.

By Mark Blumenthal on December 25, 2008 7:24 AM | Permalink | Comments (1) | TrackBacks (0)

November 4, 2008

Our 2008 Election Night Map

By Mark Blumenthal

As you may have noticed, we have inserted a new map at the top of our main page. The classic maps for President, Senate, Governor and House races are still there, just use the "map chooser" pull-down menu to see them. We will be live blogging tonight, starting a little after 5:00 Eastern Time.

Here is a quick description of the numbers in the map and where we get them.

First, we will be monitoring the five major television networks and the Associated Press and tracking the calls that each makes in the race for President. Once one network projects a state for a candidate, we will color the state light red or light blue for McCain or Obama. When all six organizations have made their projections, we will change the map to a darker shade of red or blue. States where the polls have closed, but where it is still too early or too close to call, will be colored yellow (also the color we will use in the extraordinarily unlikely event that any of the networks make a conflicting call).

To see which networks have made their projections, just point your cursor at the state to see an expanded "tool-tip" displaying that information. The tool-tip for each state will also display two important columns of data:

Pollster Trend -- The far right column will display our most recent trend estimate based on the pre-election polls. And just like our standard map, you can click on the state to display our chart for that state. See our map FAQ for more information on how we compute our trend estimates.

Est Result -- Shortly after the polls close, we hope to display the "estimated result" of the vote shares in each state culled from the network exit poll tabulations posted online (by CBS, CNN, Fox and NBC). These tabulations show the exit poll results by demographic and other subgroups (age, race, party, etc.). We will extrapolate the underlying vote estimates used to weight each table and display these in the Estimated Result column on the tool-tip.

During the course of election day and evening, the people who run the exit poll and projection operation have various estimates of the outcome in each state, estimates that gradually improve as they obtain first exit poll interviews, later the actual vote cast in random samples of precincts, and ultimately the actual vote count. When the polls close, and at two or more additional times during the night, the analysts will re-weight the tabulations based on more current and accurate estimates.

Important disclaimer: These estimates are most likely not undiluted "exit poll" results. At poll closing, the exit poll tabulations that appear online are most likely weighted to a "composite estimate" that averages the results of exit poll interviews with the averages of pre-election polls (not at all unlike the trend estimates we post here at Pollster.com). Also, as we learned during the primaries, the weighting of the cross-tabulations frequently falls far behind the up-to-the-minute estimates that network "decision desk" analysts use to call the race.

Note that we have added separate labels for the individual congressional districts of Nebraska and Maine, since these states allocate Electoral Votes partially by district and may split their electoral votes. Unfortunately, we were only able to obtain public polls for Nebraska-02, so that label is the only one of the districts that will click-thru to a chart.

By Mark Blumenthal on November 4, 2008 3:53 PM | Permalink | Comments (15) | TrackBacks (0)

November 3, 2008

A Thank You or Two...and Some Housekeeping

By Mark Blumenthal

First, a very personal thank you. I was surprised and deeply gratified by the response to my post a week ago about the death of my father-in-law, both in the comments and by email. I apologize for not responding to every note personally -- I am hoping to do so after the election. My wife's family has, for most of the last week, been practicing the Jewish ritual of Shiv'ah, and I have frankly struggled to balance my obligations to family and those to this site during the final, incredibly busy week for which we have prepared for the better part of two years. So your kind words have been a great comfort.

More important, those who left comments should know that without realizing it, paid your own virtual visit to the home of the Burstin family and thus did what Jews consider a great "mitzvah" (a good deed commanded by God). On Tuesday night, following the funeral, I shared my post with my wife and my brother-in-law who had been, up until then, understandably preoccupied with other matters. The immediately scrolled down to read the comments and were visibly moved by the outpouring of kindness shown by so many strangers who never knew their father. So please accept my thanks on their behalf as well (the most appropriate place to make contributions in Frank Burstin's would be the United States Holocaust Museum).

Second and more generally. We quietly achieved the milestone of a million page views about a week ago and have served over 1.2 million pages for five of the last six days. During October, we had over 23 million pages views and 1.9 million absolute unique visitors. I find that level of traffic truly mind boggling, and it is a big reason why I have been so committed to working and posting over the last week. Thank you for your confidence.

We realize that most of you are experiencing a unique, one-every-four-year obsession with polls and polling data, so we have no illusions about where the traffic will head after Wednesday, but we will still be around and have plans for aggregating, charting and analyzing public opinion more broadly as we move into a new presidential administration next year. We hope you come back and check in on us from time to time.

Meanwhile, a few "housekeeping notes." We apologize for the slow down many of you experienced this morning that seemed to peak about noon eastern time. In reaction to an unusually heavy load of traffic on our servers, our IT support staff made some changes to the way our computers are configured which appears to have eliminated most of the slow down.

Some of you also emailed to report a minor glitch affecting the charts that made the trend line appear to turn back on itself slightly in a few instances that was visible only when you focused on just the last month or two on the trend line. Our Flash developer quickly smashed the bug and we uploaded a new version of the chart program that should solve the problem (though you may need to clear your browser cache and reload the page). If you are still seeing the problem or any other glitch, please drop us an email.

Also, we quietly added a feature last week that some of you will find helpful over these last 24 hours. The main "Poll" pages for Polls on the races for President, Senate, Governor and U.S. House now feature tables showing the current trend estimates and classifications for all races, including all 107 House races for which we have data (some of which have been added too late to be included on our House map). An undocumented tip: You can easily copy and paste those tables into a spreadsheet for sorting or further manipulation.

Stay tuned for more tomorrow: We will be live blogging about the results tomorrow night and will have a special, expanded election night map with results and network calls.

And finally, a request I hope offends no one. I'm going to add the "donate" button to our front page and side bar so, if you have enjoyed this site and would be willing, you can make a contribution and help us both build a better site for the future and help me give an end of cycle bonus to Eric and others who have worked very hard over the last two years to bring you the charts and data every day.  (And no, donations are not tax deductible). 

By Mark Blumenthal on November 3, 2008 5:27 PM | Permalink | Comments (28) | TrackBacks (0)

October 30, 2008

Comments are Back On

By Mark Blumenthal

I have restored the comments function I temporarily disabled earlier this afternoon. Those who continue to post abusive or profane commentary will be banned without warning, as our time allows. If you are in doubt, please read my post from earlier today.

If the tone reverts to the out-of-control ugliness I have seen in recent weeks, we may shut the comments off altogether for the remaining days before the election.

By Mark Blumenthal on October 30, 2008 6:56 PM | Permalink | Comments (82) | TrackBacks (0)

Why Are The Comments Disabled?

By Mark Blumenthal

So why aren't the comments working this afternoon? I temporarily disabled them. Why?

Let me explain. We have always considered it important to maintain a largely unmoderated comments section that allows for dissenting views and debate over the topics raised by each post. Under the right circumstances a community of commenters forms that will help maintain a mostly civil forum for the expression of dissent and add great value to what we post here.

In recent days, I have seen some very impressive examples of our comments section functioning exactly as it should. Last Friday, I posted a lengthy entry that discussed likely voter models. It generated many comments. Some dissented from my argument or questioned some aspect of it, some added thoughts or theories of their own. And while some disagreed with each other, the comments that I read were generally civil, respectful and connected to the topic at hand.

Monday I posted a very personal note on the passing of my father-in-law. The many comments that followed were moving and beautiful. The outpouring restored my faith in the idea of an open, mostly unmoderated comments section (and thank you, thank you to all who posted so many kind words -- it meant a great deal to my family).

And then there are the comments left on our "poll update" posts that have degenerated into something altogether different. And that is partly my fault.

We do have a basic comments policy that requires, simply, that commenters "keep the dialogue civil." It also warns that "comments that we consider abusive, profane, hateful or racially, ethnically or otherwise objectionable" are subject to deletion, and the reader that post such comments are subject, at our discretion, to being banned from commenting on this site.

Until this past summer, I forwarded every comment left on this site to my email inbox, and made it a policy to read (or at least skim) every comment. Occasionally, someone would post an abusive or overtly profane comment, and would delete it. A small handful of commenters were so brazen about ignoring the rules that we banned them.

Unfortunately, when the volume of comments started to exceed a hundred or more every day, I could no longer keep up with it and to be honest, at that point, things started to get out of control. We are now getting more than a thousand comments a day -- in the last week, we received more than 10,000. At that pace, given our modest resources, it is simply impossible to read every comment, much less try to monitor or police them.

And unfortunately, the level of abusive, insults and profanity has grown to an embarrassing level. Two days ago, I received a email from a father of a 2nd grader. He wanted to know if we offered a "kid friendly" version of Pollster:

[My child's] school wants to share the site with the rest of the students. The only problem is that some of your visitors can be quite cantankerous with one another in the comments sections. Is there any way to disable those on our end? Any ideas or suggestions on how the school can use your site in a way that is appropriate for young kids?

No, we do not have a way to offer child friendly version of pollster, but I do not understand why the adults who use this site and comment on it cannot find a way to act like adults. This is not a locker room and not a night club. We have a simple policy, and the adults that comment here ought to find a way to follow it or leave. The alternative is that we disable comments altogether, just as we have this afternoon.

Yesterday, a very frequent commenter posted a comment that certainly offended me, and several other readers who emailed in protest. It said, in reference to Barack Obama, "the American Public doesn't want a Jew-hating Socialist running the economy." Now while I find that comment extremely uncivil and offensive, some might see it as a contrary opinion. So I posted a comment of my own asking the commenter to explain how that remark qualifies as remotely civil and intelligent and why I should not consider it a violation of our comment policy.

He ignored my question and instead posted a series of comments this morning including this charming response to another reader on another subject: "You STUPID liberal f*ck."

I asked a second time for some explanation. I heard none. He refused to answer my question and told another reader that he has nothing to explain. As such, he is free to take his comments elsewhere. As of today, he is banned and no longer welcome to comment on Pollster.

Now I understand that a lot of obnoxious, offensive, petty name calling has been going on in our comments section for months, and that this particular commenters behavior is just par for the course. I recognize that I bear some responsibility for letting it get out of control, but while I want to clean it up, I have no interest in who said what first or why. Since it seems to be hard to get the attention of some of you, I have shut off our comments for the afternoon. I will turn comments back on in a few hours, but before I do I want to make a few things clear:

1) If you can't say it on broadcast television, please don't post it here. Is that so hard? If you can't act like an adult when you comment, please take it somewhere else. I have only banned one commenter today, but there are obviously many others who have gotten into the habit of ugly, profane rants directed at other readers. These need to stop. Today. Those who ignore this plea when the comments come back on may find themselves locked out.

2) Don't pick an a screenname that is, itself, profane or abusive of other commenters. Doing so is grounds for being banned.

3) Banned users are banned permanently. They are not permitted to return under a new screen-name. Where possible, we will take action against those who violate this rule, including contacting webmasters or postmasters at the ISPs or businesses where the comments originate.

Now I would like to promise that Eric or I could spend every moment of the next five days carefully monitoring the comments and enforcing these rules, but will obviously be impossible. So I want to make a plea to the readers that care about our comments section and want to keep it open Be a community: Help us convince the others to clean up their act. If someone says something offensive when the comments come back on, please try to convince them to apologize and stop. If some continue to flout these rules when the comments come back, then please email us to nominate those who who deserve to be banned. However --and note this well -- please follow guidelines (borrowed from the DailyKos policies for their "Hide Ratings"):

• Do not request that we ban someone for expressing contrary opinions, so long as they do so in a civilized fashion.

• Do not request that we ban someone you are actively having a fight with.

• Please understand that we won't have time to respond personally to email received this week or to resolve disputes, we decide what we consider offensive and all decisions are final.

We have precious little time over the next 5 days and I have no sense of humor about the continuing abuse of this site and its readers. Shutting off comments altogether remains a real option, so please help us out.

By Mark Blumenthal on October 30, 2008 1:41 PM | Permalink | Comments (0)

October 22, 2008

Comments Problem

By Mark Blumenthal

For those who have tried to post comments this morning and received the following message:

Comment Submission Error

Your comment submission failed for the following reasons:

You are not allowed to add comments.

You have not been banned. The problem, which we are looking into, appears to have blocked all comments since a little after 10:00 a.m. Eastern Time.  

We apologize for the inconvenience and will update with more information when we have it. 

Update:
The bug has been solved and our comments feature is back and running.

By Mark Blumenthal on October 22, 2008 2:00 PM | Permalink | Comments (12) | TrackBacks (0)

October 6, 2008

Embed Our Electoral Vote Map

By Mark Blumenthal

I'll have more details in tomorrow morning's update, but the most obvious news tonight is that our electoral vote count as displayed at the top of our maps finally caught up with the shift in the national trend: Obama now holds a 296 to 163 advantage over John McCain, as both Colorado and Florida shift to the "lean" Obama.

So for at least some, this may be the ideal time to offer you an added feature:  You can embed a version of our small map like the one below on your own blog or website.

Just select and copy all of the code in the box below and paste it into your blog or web site (using the HTML mode):

By Mark Blumenthal on October 6, 2008 8:34 PM | Permalink | Comments (5) | TrackBacks (0)

Pollster.com via iPhone

By Mark Blumenthal

Thanks to our partners at Slate.com, you now have an easy way to check the latest Pollster.com trend estimates using Apple's iPhone.

Here are the details from Slate:

Today Slate introduces Poll Tracker '08, an application that delivers comprehensive up-to-the-minute data about the presidential election to your iPhone, iPhone 3G, or iPod touch. Using data from Pollster.com, the Poll Tracker '08 delivers the latest McCain and Obama polling numbers for every state, graphs historical polling trends, and charts voting patterns in previous elections. Poll Tracker '08 allows you to sort states by how contested they are, how fresh their poll data is, or how heavily they lean to McCain or Obama.

You can download Poll Tracker '08 on the iPhone App Store. It costs just 99 cents, a small price to pay for satisfying your craving for data anytime, anywhere. Get it on the App Store.

By Mark Blumenthal on October 6, 2008 1:45 PM | Permalink | Comments (9) | TrackBacks (0)

House Race Data (and Map!)

By Mark Blumenthal

Yes, we have U.S. House race data! Actually, we have posted charts for House races for some time, but on Friday we finally put up our revised House map (it's accessible via the "map chooser" pull down from any of the large maps on the site) and added that page to our main menu.

Readers should know that public polling data for U.S. House races tends to be more rare. In 2006, we scoured various sources for poll in House races and ultimately found polls in just 94 of the 435 districts. This time, we created a map with labels for the 111 districts rated as competitive or potentially competitive by our colleagues at the Cook Political Report. So far, we have logged in poll results for 60 districts, with another dozen or so on the way this week.

A note of explanation about the map and House scoreboard: Where we have poll data, we classify the race based on the polling data using the same criteria as for the statewide races. Where we have no polls at all, we assume no change in party status.

We assume there are U.S. House surveys out in the public domain that we no nothing about. So if you know of a poll in a U.S. House race that's not listed here, please email us (questions at pollster dot com) with the details. Thank you!

By Mark Blumenthal on October 6, 2008 12:36 PM | Permalink | Comments (2) | TrackBacks (0)

October 1, 2008

Washington Post Live Chat

By Mark Blumenthal

I totally neglected to link to this -- apologies for that: Charles Franklin and I are joining Nate Silver of FiveThirtyEight.com for a live chat on WashingtonPost.com. The chat starts right now (noon Eastern Time) but should be available for review afterwards.

By Mark Blumenthal on October 1, 2008 12:03 PM | Permalink | Comments (7) | TrackBacks (0)

September 26, 2008

New Charts for Polls, Polls, Polls

By Mark Blumenthal

A little over two years ago, we launched Pollster.com with a mission of providing a complete compilation of poll results, expert analysis and graphical tools to help readers make sense of polling data. Today, after two long years of development, our commitment to interactive graphical tools takes a quantum leap.

At a moment when the political world is swimming in a flood of polling data, we are pleased to announce a new, fully interactive Flash chart application that will plot all of the poll charts here on Pollster.com. The new charts allow you to:

• Select or limit the polls used to draw trend lines and calculate polling estimates with the "filter" tool. If you don't like a particular pollster, just un-click and take them out (yes...really).

• Toggle between the display of the default trend line and alternatives that are more or less sensitive using the "smoothing" tool -- these are essentially the same as the "steady blue" and "ready red" trend lines often used by Charles Franklin.

• Hold your mouse over any data point to display details about each the poll.

• Click the mouse on any data point to "connect the dots" between all polls fielded by that pollster.

• Modify the date range (x-axis) and percentage range (y-axis) by clicking on either axis directly or with forms found on the "tools" menu.

• Select the candidates you want to see displayed on the chart with the "choices" tool .

• Toggle the display of data points, trend lines and grid lines on or off with the "plot" tools.

• Copy the code necessary to bookmark your customized chart or share it via email with the "URL" tool.

• Get the code necessary to place a small version of the customized chart on your own blog or web site with the "Embed" tool.  [Update: We believe we've squashed the embed bug.  If you experience problems with the embed tool or anything else, please email us with details at questions at pollster dot com.  We missed a bugin the embed function that prevents the embedded chart from displaying customized filters.  Apologies -- we should have this cleaned up soon]
  

As of this posting, we have converted our charts for all presidential charts, including Ohio, Colorado, Virginia , Florida, Michigan, Pennsylvania, New Hampshire, Nevada, Minnesota and the National Trend to the new chart, and we should have most of the presidential charts converted over the course of the day. Although we hope you will dive right in and try these new features, we have also created a quick video guided tour of some of the most important features.

We are really excited about the possibilities these new tools create for poll junkies to explore and discover the wealth of data now available and how you are too. So if you like If you like what you see, we hope that you share this news with your friends. If you have a blog or diary, we would very much appreciate a link to this entry and please try out the embedding feature to see how it works. I'll be back later with some extra tips on how to use the charts. Meanwhile, please don't hesitate to email us with your questions or reactions.

Where credit is due: A lot of very talented people worked exceptionally hard to make these new charts possible, but most deserving of thanks are Quentin Fountain and Technorganix (for Flash design and development), Jeff Lewis and Seth Hill (for development of the underlying database and statistical architecture), Charles Franklin (for the design of the original charts and the regression trend lines and smoothing routines) and, last but not least, Eric Dienstfrey, who has entered virtually every piece of the data now displayed on Pollster.com and is doing the nitty gritty work of implementing these charts and keeping Pollster operating every day.  Update: I nearly forgot to thank those of you -- and you know who you are -- who helped beta test these charts over the past week.

By Mark Blumenthal on September 26, 2008 8:09 AM | Permalink | Comments (52) | TrackBacks (0)

September 25, 2008

Internet Service Interruption

By Mark Blumenthal

Sorry to all for the slow pace of updates today. We have had an Internet service interruption that's kept us offline for much of the afternoon. We seem to be back online...for the moment at least.

By Mark Blumenthal on September 25, 2008 3:01 PM | Permalink | Comments (32) | TrackBacks (0)

September 9, 2008

Introducing Politics Home

By Mark Blumenthal

If you are a regular reader, you no doubt noticed the new PoliticsHome.com panel that now appears in the right column on every page on Pollster.com.

PoliticsHome.com launched in April 2008 in the United Kingdom and has quickly established itself as a leading resource for constant updates on British politics. This past week, PoliticsHome kicked off Campaign08, a new site devoted to U.S. presidential election news.

Here is how it works: PoliticsHome.com is modeled on the financial news services used by traders that provide links and headlines -- "everything you need to know, each minute" -- on single screen that updates constantly throughout the day. They have a team of political journalists in bureaus in London and Washington that constantly review political news, publishing full news digest each morning at 6 a.m. Eastern time, with live updates until midnight.

They are launching the new site in association with Pollster.com. We will provide PoliticsHome with regular polling updates, and they have created a miniature version of the Campaign08 site that now runs in the right column on every page on Pollster.com. The "Latest Developments" box gets the same minute-by-minute updates as their main site. So now, you can visit Pollster.com for both the latest poll results and a quick, constantly updating digest of all the breaking political news across the web.

PoliticsHome.com will also be introducing some data collection efforts in the coming weeks that will be of interest to Pollster.com readers, so stay tuned.

By Mark Blumenthal on September 9, 2008 3:03 PM | Permalink | Comments (1) | TrackBacks (0)

August 28, 2008

Convention Odds and Ends

By Mark Blumenthal

First, a quick update on yesterday's post on the Obama campaign briefing. First, James Barnes of the National Journal has a write-up of the strategy spelled out by campaign manager David Plouffe yesterday that includes extended verbatim excerpts from the briefing.

In the briefing, Plouffe emphasized that when it comes to polling "all we care about is these 18 states." I had not specified those 18 states, but they are: Alaska, Colorado, Florida, Georgia, Iowa, Indiana, Michigan, Missouri, Montana, Nevada, New Hampshire, New Mexico, North Carolina, North Dakota, Ohio, Pennsylvania, Wisconsin, and Virginia.

Second, on Tuesday, I participated in a panel on polling put on by the National Journal along with Hotline editor Amy Walter and pollsters Ed Reilly, who conducts the Diageo Hotline poll and Geoff Garin, who worked for Hillary Clinton earlier this year. Video excerpts from that panel are now available at the this link.

Finally, in an hour or so, I will be going offline and heading over to Invesco Field for the evening. I've been experimenting with Twitter this week (under the handle of, what else, MysteryPollster), and will probably post some comments there tonight (with the caveat that any such "tweets" are likely to be a bit off our usual focus on polling and surveys).

By Mark Blumenthal on August 28, 2008 3:08 PM | Permalink | Comments (9)

August 17, 2008

A Break

By Mark Blumenthal

I will be taking a break this week, although I filed a National Journal column for the week, which should appear in a day or two. Meanwhile, Eric will continue with poll updates and our other contributors should be active this week. See you next week!

By Mark Blumenthal on August 17, 2008 9:49 PM | Permalink | Comments (0) | TrackBacks (0)

August 13, 2008

How We Choose Polls to Plot: Part III

By Mark Blumenthal

In the first two installments of this online dialogue, I asked a question we have heard from readers about why we choose the results for "likely voters" (LVs) over "registered voters" (RVs) when pollsters release both. Charles answered and explained our rationale for our "fixed rule" for these situations (this is the gist):

That rule for election horse races is "take the sample that is most likely to vote" as determined by the pollster that conducted the survey. If the pollster was content to just survey adults, then so be it. That was their call. If they were content with registered voters, again use that. But if they offer more than one result, use the one that is intended to best represent the electorate. That is likely voters, when available.

Despite my own doubts, I'm convinced by the rule for this reason: I can't come up with a better one. Yes, we would arbitrarily choose RVs over LVs until some specified date, but that would leave us still plotting numbers from pollsters that only release LV samples. And on which date do we suddenly start using the LV numbers? After the conventions? After October 1? What makes sense to me about our rule, is that in almost all cases (see the prior posts for examples) it defers to the judgement of the pollster.

Several readers posed good questions in the comments on the last post. Let me tackle a few. Amit ("Systematic Error") asked about how likely voters are constructed and whether we might be able to plot results by "a family of LV screens (say, LV_soft, LV_medium, LV_hard)" and allow readers to judge the effect.

I wrote quite a bit back in 2004 about how likely voter screens are created, and a shorter version focusing on the Gallup model two weeks ago. One big obstacle to Amit's suggestion is that few pollsters provide enough information about how they model likely voters (and how that modeling changes over the course of the election cycle) to allow for such a categorization.

"Independent" raised a related issue:

Looking at the plot, it appears that Likely Voters show the highest variability as a function of time, while Registered Voters show the least. Is there some reason why LVs should be more volatile than RVs? If not, shouldn't one suspect that the higher variability of the LV votes is an artifact of the LV screening process?

The best explanation comes from a 2004 analysis (subs. req.) in Public Opinion Quarterly by Robert Erikson, Costas Panagopoulos and Christopher Wlezien. They found that the classic 7-question Gallup model "exaggerates" reported volatility in ways that are "not due to actual voter shifts in preference but rather to changes in the composition of Gallup's likely voter pool." I also summarized their findings in a blog post four years ago.

Finally, let me toss one new question back to Charles that many readers have raised in recent weeks. The two daily tracking surveys -- the Gallup Daily and the Rasmussen Reports automated survey -- contribute disproportionately to our national chart. For example, we have logged 51 national surveys since July 1, and more than half of those points on the chart (27) are either Gallup Daily or Rasmussen tracking surveys. Are we giving too much weight to the trackers? And what would the trend look like if we removed those surveys?

By Mark Blumenthal on August 13, 2008 2:26 PM | Permalink | Comments (3) | TrackBacks (0)

August 12, 2008

How We Choose Polls to Plot: Part II

By Charles Franklin

By Charles Franklin on August 12, 2008 10:16 AM | Permalink | Comments (12) | TrackBacks (0)

August 11, 2008

How We Choose Polls to Plot: Part I

By Mark Blumenthal

Since adding the maps and upgrading this site, we have received a number of good questions about how the charts and trend lines work and why we choose to include the poll results that we do. I want to answer a few of those questions this week before we call get swept up in the conventions and the final stretch of the fall campaign.

Our approach to charting and aggregating poll data follows the lead and philosophy of our co-founder Charles Franklin. And while I am tempted to describe that approach as well entrenched, the reality is that in many ways it has and will continue to evolve.

Since launching this site nearly two years ago, Franklin and I have continued to discuss (and occasionally debate) some of the technical issues offline. Most of the time we agree, but I tend to propose ways to change or tinker with our approach, and Franklin usually succeeds in convincing me to stay the course.

In considering some of issues that came up more recently, I thought it might be helpful to take this dialogue online. Hopefully, we can both answer some of the questions readers have asked and also seek further input on those issues we have not completely resolved.

So with that introduction out of the way, here is the first question for Franklin:

Over the last few weeks, in commenting on the "likely voter" subgroups reported by Gallup and other national pollsters, I have essentially recommended that we focus on the more stable population of registered voters (RV) now, and leave the "likely voter" (LV) models for October (see especially here, here, here and here). Yet as many readers have noticed, when national surveys publish numbers for both likely and registered voters, our practice has been to use the "likely voter" numbers for our charts and tables.

Why?

Like the other sites that aggregate polling results from different sources, we face the challenge of how to best choose among many polls that are not strictly comparable to each other. Even if we examine data from one pollster at a time, we will still see methodological changes: Many national pollsters will shift at some point from reporting results from registered voters to "likely voters." Some will shift from one likely voter "model" to another, or will tinker with the mechanics of their model, often without providing any explanation or notice of the change. And no two pollsters are exactly alike in terms of either the mechanics they use or the timing of the changes they make.

As such, two principles guide our practices for selecting results for the charts and tables: First, we want to defer to each pollster's judgement about the most appropriate methodology (be it sample, questionnaire design or the most appropriate method to select the probable electorate). Second, we want a simple, objective set of rules to follow in deciding which numbers to plot on the chart.

In that spirit, when pollsters release results for more than one population of potential voters, our rule is to use the most restrictive. So we give preference to results among "likely" voters over registered voters and to registered voters over results among all adults. In almost all cases, the rule is consistent with the underlying philosophy: The numbers for the more restrictive populations are usually the ones that the pollsters themselves (or their media clients) choose to emphasize.

But there have been some notable exceptions recently, of which, last month's ABC News/Washington Post poll provided the most glaring example. ABC News put out a report and filled-in questionnaire with two sets of results: They showed Barack Obama leading John McCain by eight points (50% to 42%) among registered voters, but by only three points (49% to 46%) among likely voters. Following our standard procedure, we included the likely voter numbers in our chart.

However, ABC News emphasized the eight-point registered voters numbers in the headline of their online story ("Obama Leads McCain by Eight But Doubts Loom"). Within the text, they first reported the registered vote numbers and then used the likely voter results to argue that "turnout makes a difference." The 8-point lead also made the headline of the Washington Post story, but they did not report the likely voter results at all, either in the text of the story on in their version of the filled-in questionnaire.

So in this case, the news organizations that sponsored the poll clearly indicated that the RV numbers deserved greater emphasis, yet we followed our rule and included the LV numbers in our charts.

Charles, in cases like these, should we find a way make an exception? And why not just report on "registered" voters until after the conventions?

Update: Franklin answers in Part II.

By Mark Blumenthal on August 11, 2008 5:59 PM | Permalink | Comments (3) | TrackBacks (0)

July 25, 2008

Housekeeping: Our Classifications

By Mark Blumenthal

As of this morning we have made some small changes to the algorithm that classifies leaders in each state on the maps we display for the Presidential, Senate and Gubernatorial races. As quite a few readers noticed earlier in the week, there were some odd inconsistencies in the way the margins separating the candidates translated into "toss-up" or "lean" status. So we have changed the classification slightly to make the process more consistent and intuitive.

The bottom line is that the new criteria shifts three states into the yellow toss-up designation, Virginia, North Carolina and Arizona (more about that last one below). Also, two states that had been designated strong Obama (Michigan and Iowa) are now lean, while Georgia moves from strong to lean McCain.

The inconsistency between states was partly technical (having to do with the way we calculated the margin between the candidates) and partly a function of calculating a confidence interval around the trend line. That process meant that states with more polls in their chart were producing much narrower "margins of error" than polls with few.

So we have changed to using a confidence interval based on the average sample size for the available polls in each state (essentially the same approach we used during 2006). That means that the margin necessary to be classified a leader will be slightly smaller in states where pollsters tend to conduct more interviews (such as Pennsylvania, Connecticut and Texas). But overall, things will be more consistent.

This choice may tend to overstate the uncertainty about a leader in some cases, especially where we have a huge number of polls (such as the National trend). Given all of real-world variability (and volatility) built into opinion surveys, particularly the difficulty of approximating the "likely" electorate at this stage in the contest, we believe it is better to err on the side of less certainty rather than more.

We are also using a totally subjective standard which will on occasion produces some strange results. We believe the "smoothed" loess regression trend lines that appear in the charts provide the best visualization of the trends that underlie available data. It also has the very helpful feature of essentially ignoring "outlier" values in most instances. One poll in ten, no matter how out of line with the trend, rarely budges the trend line.

But there is one circumstance that can produce squirrelly results: When a new "outlier" poll appears in a state with relative few polls, especially if several weeks or months have passed since the last poll.

That, unfortunately, is exactly what happened in Arizona, not once but twice in late June. First, the ASU/Cronkite released a survey based on 175 interviews showing 34% of registered voters as undecided and producing much lower than usual percentages for both McCain and Obama. Next, the always shaky Zogby Internet Panel survey showed a three point Obama lead (and included Barr at 7%). The combination has narrowed the McCain's trend-line margin to just 3.5 percent, just enough to edge below the "toss-up" line.

If we throw out the Zogby poll, the trend lines would now show McCain leading by 10 percentage points. If we left the Zogby poll in, but threw the ASU/Cronkite poll out, McCain would lead by more than 6 points. Either margin would easily classify Arizona as lean McCain. The next poll out in Arizona will most likely have the same effect.

And as long as we are on the subject of housekeeping, please note that by popular demand we added back a "blog roll" of links to all 50 state charts and all of the charts with available data in race. That collection of links now appears on our front page and everywhere else on Pollster.com. So if you are not a fan of the maps, or just prefer to jump from chart to chart with one click, the links are back in place.

By Mark Blumenthal on July 25, 2008 11:59 AM | Permalink | Comments (8) | TrackBacks (0)

July 23, 2008

New Contributors

By Mark Blumenthal

Yesterday, I explained some of the changes to the structure of Pollster.com. Today I want to tell you more about some of the new names that we have added to our menu of regular contributors.

The first two should be familiar. Democratic pollster Margie Omero has been a regular contributor since her first "guest pollster" item here last November. She has been understandably busy conducting surveys for her clients in recent months, but we look forward to her continuing presence here.

David W. Moore has been contributing items more recently. You may recognize Moore as a former Managing Editor of the Gallup Poll or as an author of several books on polling and pollsters. His latest book, The Opinion Makers, is forthcoming in September. His posts here will also appear on his blog, Skeptical Pollster.

Two names are new to our site but not to true political junkies. We are very pleased to add Republican pollster Steve Lombardo, the president and CEO of Lombardo Consulting, as a regular contributor. His weekly email update, the LCG Election Monitor, is well known to political journalists and insiders as a source of straight-shooting analysis of political poll trends. Starting today, the LCG monitor will also be published every week right here on Pollster.com.

Finally, regular Pollster readers will probably recognize Brian Schaffner, professor of political science at the University of Massachusetts, Amherst, from my regular links to his posts on the CCPS blog earlier this year. While his day job does not allow for the sort of daily blogging he managed this spring, we look forward to once or twice a month contributions from him here.

And finally, speaking of academics with day jobs, Pollster.com's co-creator Charles Franklin continues to be an integral part of this site. He has been busy this summer doing the work the University of Wisconsin pays him to do, but his regular contributions should return soon.

As you may have noticed already, the name of each now appears in the "Analysis" menu at the top of every page. Choose a name from the menu to see only the posts from that person. We have also created RSS feeds specific to each contributor that are available here.

By Mark Blumenthal on July 23, 2008 4:47 PM | Permalink | Comments (1) | TrackBacks (0)

July 22, 2008

How to Find Charts on the New Pollster.com

By Mark Blumenthal

Apologies for not posting this item sooner. Unfortunately, we ran into some unexpected problems this morning with the update. Among other things, we were not able to update the site until a few hours ago, so I was not able to post any sort of explanation of what has changed at Pollster.com, and what changes are still in store.

So let me start by reviewing the most pressing issues.

First, we know that comments were not working for much of the day today. We believe they are functioning properly now, but please email us if you experience any problems posting comments to the new site.

Second, while we think most readers will find the Flash maps provide the most convenient way to navigate the site, we overlooked the need to provide a bit more guidance on how to navigate in our push to get things up and running. We consider today's launch the first step in an ongoing upgrade process that will continue between now and late August. So if you have specific complaints or suggestions, please leave a comment below or send an email (though you might want to read the rest of this post before hitting the send button).

How Do I Find a Chart?

We believe the Flash maps that appear on our home page and elsewhere on the site provide the most convenient means of navigating to our charts. The maps do require the Adobe Flash Player plug-in for your browser (download it here, details on how to navigate without the Flash maps below).

The map on the main page initially displays results for the presidential race. Point your mouse at a state and the most recent trend estimate numbers pop-up small "tool tip" window (more details on how we generate the numbers and classify leaders here). Click on any state to go directly to our chart of the presidential poll results for that state.

If you want to find polls for other races use the "Map Chooser" pulldown in the upper left corner of any map and select another category: Senate or Governor (we will add a map for U.S. House races in August). Then follow the same procedure to click through to a state. The links will be active if we have poll data available.

Looking for a poll for a particular state from 2006, from the 2008 primaries or for the now inactive Clinton-McCain general election match-ups? To find these with the map, use the "Find All Polls" option to display the all-gray map. Click on any state to navigate to the index page for that state that lists links to every chart page ever created on Pollster.com (as a text-HTML link) for that state. Click on the small US icon in the lower right corner of the chart to display the index of all national trend charts (including Bush job approval).

We designed the state index pages to work automatically, so that new links will always appear there as we add new charts to the site.

But what if I cannot run Flash and cannot see the maps?

The main menu at the top of the page to the right of the (new) Pollster.com logo has an item named "The Polls." Hold your mouse over "The Polls" and a drop down menu will appear with links to main pages for various categories: President, Senate and Governor. Text links to state level charts are available on each page.

In other words....

To access the text links to the Presidential chart pages, choose "President" from "The Polls" menu. Those text links now appear below the map.

To access the text links to races for Senate or Governor (including data from 2006), click the appropriate category on "The Polls" menu and you will see a list of text links to those pages as well.

Also, you can click directly on "The Polls" in the main menu to go to a page of the same name that has text links to each of the 50-state index pages (listing all poll charts for each state) and also links to national trend charts that appear just below the map.

Where is the national Clinton-Obama trend? It's there. Click the link to the left. To navigate to any national level trend data, use the small USA icon in the bottom right of each map, or (for text links) click on "The Polls" on the main menu to get to our main Polls page.

What happened to the Bush job approval chart? It too is still there, as are the other national trends we posted on the old site. Links to all national trend data are available are on the main "Polls" page.

I will take full responsibility for temporarily pulling the Bush job approval from the main page. Charles Franklin has kindly maintained and updated those charts, but his day job understandably prevents more frequent updates. We are working now to bring his seven years of job approval data into our back-end database so that we can update the job approval chart whenever a new national poll is released. We hope to have that system in place during early August.

Again, today's launch is just the first step. We have more in the works, although today's changes are probably the most dramatic. We have made changes already today in response to the comments we received via email and welcome your suggestions for how to make this site better.

I will have more to say about changes on the blog side of Pollster a bit later.

By Mark Blumenthal on July 22, 2008 6:41 PM | Permalink | Comments (7) | TrackBacks (0)

May 23, 2008

Programming Help Needed

By Mark Blumenthal

Are you a programmer skilled in some combination of PythonSQL, CherryPy, R-Statistics or Flash Graphics? If so, are you also a student looking for a summer gig or just someone looking to pick up some part-time freelance projects? Would you like to help Pollster.com be all that it can be?

If so, we'd love to hear from you. To be clear, we do not have a full-time position to fill. Rather, we are seeking part-time help, especially over the summer months, with several development projects. If you're interested, please send a resume and email describing your abilities and availability to questions at pollster dot com (please use the subject heading "Programming Help").

By Mark Blumenthal on May 23, 2008 5:07 PM | Permalink | Comments (1)

May 13, 2008

Why No Summary Numbers on the West Virginia Chart?

By Mark Blumenthal

Pollster reader Dave asks:

Why have you stopped including summary numbers in your graphs?

As most of you know, our charts usually include a regression trend line (a line drawn through the points) as well as a legendd at the top of each chart that includes the most recent value of the trend estimate for each candidate. This example below shows the final candidate estimates for the Indiana Primary:

However, as Dave and other alert readers have noticed, the charts for West Virginia and all of the remaining primary and caucus states except Oregon no longer display any sort of "summary number." Their legends look like this:

The reason is that we do not plot a regression trend line when we have less than eight polls. Why not? Here is Professor Franklin's explanation posted when we started running these charts last year:

Ideally, the trend estimator should have a dozen or more polls to work with before we take the trend very seriously. When the number of polls drops too low, the trend estimator will jump around considerably if new polls are very far from previous polling and may produce jagged trend lines that are likely to change with more data. Despite this danger, we've estimated the trend with as few as eight polls, rather than stick to the safer minimum of twelve. Too many states are between 8 and 12 polls to ignore, and while we are cautious most of the trends look pretty reasonable even with less than 12 polls. When the polls are consistent with each other, the trend estimate will still be pretty good even with eight. But when there is substantial disagreement among the polls, we will get jagged or otherwise "bad" trends. Since the plots show the actual polls you can look at the data yourself and decide whether the trend is a reasonable fit to the data, or if it is erratic enough to be discounted. You decide.

Until this past weekend, the legend on charts with less than eight polls had displayed the candidate numbers for the median poll (or the average of the two middle polls when we had 2, 4 or 6 polls available. While that approach made sense to us last year, we found two practical problems: (a) the median poll was sometimes very outdated and (b) nearly everyone was confused by this approach. Many concluded the numbers in the box were clearly "wrong" as they didn't seem to match the dots on the chart. So rather than adding a confusing explanation in small print, we thought it better to err on the side of caution and simply drop the "summary numbers" when we have fewer than eight polls.

As of this morning, the number of available polls is just shy of the magic number 8 in West Virginia (6) and Kentucky (7) and a long way off in Montana (1), South Dakota (1) and Puerto Rico (1).

For those who must have a summary number, the average of the last four polls for West Virginia as of this writing (all conducted in the last two weeks) is Clinton 61%, Obama 24%

By Mark Blumenthal on May 13, 2008 2:04 PM | Permalink | Comments (0)

April 24, 2008

Has Polling Killed Democracy?

By Mark Blumenthal

Just to make the week more challenging (in light of Eric Dienstfrey's well deserved but not so well timed vacation this week), I agreed some months ago to speak tomorrow on a special panel, "Has Polling Killed Democracy", put on by The Miller Center of Public Affairs at the University of Virginia. So posting will be delayed tomorrow. Apologies in advance for that. We will (hopefully) get back to normal on Monday.

If you are in the Charlottesville area, you can find details on the panel here (pdf) and directions to the Miller Center here. The panel is free and open to the general public.

If you are not in Charlottesville tomorrow and still have too much free time on your hands, the panel will be webcast live and archived online at the Miller Center website.

By Mark Blumenthal on April 24, 2008 3:46 PM | Permalink | Comments (2)

April 18, 2008

Why Weekend Updates May Be Slow

By Mark Blumenthal

Back in September, Eric Dienstfrey, ever the model employee, asked politely if he might be able to take a week off in April for the Passover holiday and to attend to a family obligation.

I said: "April? Please. That should be an easy month." And in a managerial move that was not the most far sighted, I told him it would be fine to take off the week off starting on April 18.

Good planning, huh?

And of course, had I thought about it for a minute, I would have remembered that I would be likely to have travel plans of my own for Passover (which begins tomorrow night), especially given that April 20 is also my parent's 50th wedding anniversary...and my wife's birthday.

So this is a long way of saying that Eric will be off for the next week and in about 15 minutes I'll going offline to enjoy all the pleasures of a six hour drive from Washington, DC to Cleveland Ohio. As such, updates will certainly be infrequent for the rest of the day and over the weekend. Knowing how closely everyone (including yours truly) is following the Pennsylvania primary, I will post and update as often as I can.

Meanwhile, for those alert readers who often email us with the latest survey, please feel free to post comments to this post today with links to any new polls (just remember, no more than 2-3 links per comment or it won't publish).

And finally, if you happen to be dining in Breezewood, PA later tonight and see a guy trying to concentrate on his laptop while his wife and two young kids eat, well, that will probably be me.

[Typos and grammar repaired - and if you see more of such errors than usual this week, you will know another reason we value Eric]

By Mark Blumenthal on April 18, 2008 2:54 PM | Permalink | Comments (6)

March 17, 2008

Typekey Login And Our Comments Policy

By Mark Blumenthal

As of today, in order to post a comment on Pollster.com, we are requiring that you log-in with a Typekey identity and that you set your Typekey preferences to share your email address with us. I posted on Saturday that this change was coming. Here are some additional details on why we are making this change and about our comment policy going forward.

First, we are requiring that you share your authenticated email address because of repeated complaints from our most loyal readers about an epidemic of abusive comments and "sock puppetry." Unfortunately, those problems have worsened significantly in recent days. We promise that (a) we will not display your email address in any form when you post, (b) we will never share your email address with anyone without your explicit permission and (c) we will not SPAM you with unsolicited mass mail (though we might want to send a personal thank-you or query on occasion).

Second, for those unfamiliar, Typekey is a free, third-party system that allows you to create a unique username. The registration is fairly painless -- you just need to be a real person (not a SPAM "bot") and leave a working email address with Typekey that can be used to validate your account. If you wish to remain anonymous, you can use a pseudonym when you comment.

Third, we consider it important to maintain a largely unmoderated comments section that allows for dissenting views and debate over the topics of interest raised by each post. Our one primary rule is that commenters keep the dialogue civil. As such, we will not hesitate to delete comments that we consider abusive, profane, hateful or racially, ethnically or otherwise objectionable, and as we deem warranted, ban the commenters that post them. To underscore: Terms like "stupid," "moron," "retard," "dumbass," "loser" and the like qualify as abusive. Clear?

Fourth, we will immediately ban commenters who use their online identity to impersonate any person or entity or who use multiple identities for a single Typekey account.

A more difficult issue raised by many of you is whether we need to moderate comments to eliminate those that stray "off-topic." While we are certainly a site where the main posts concern survey data and methodology, poll results are usually at the heart of the most contentious political debates. So we are reluctant to try to draw a line between discussions of polling data and the underlying issues and controversies they involve. However, we have limits. Please try to keep the debate civil, reasonable and intelligent.

We welcome comments or complaints via email about this policy, or any reports of technical glitches with the procedure. Hopefully, the comment bugs experienced recently are now are behind us.

Addendum - October 30, 2008

Too clarify, some additional do's and don'ts for comments:

1) If you can't say it on broadcast television, please don't post it here. Is that so hard? If you can't act like an adult when you comment, please take it somewhere else. I have only banned one commenter today, but there are obviously many others who have gotten into the habit of ugly, profane rants directed at other readers. These need to stop. Today. Those who ignore this plea when the comments come back on may find themselves locked out.

2) Don't pick an a screenname that is, itself, profane or abusive of other commenters. Doing so is grounds for being banned.

3) Banned users are banned permanently. They are not permitted to return under a new screen-name. Where possible, we will take action against those who violate this rule, including contacting webmasters or postmasters at the ISPs or businesses where the comments originate.

Be a community: Help us convince the others to clean up their act. If someone says something offensive, please try to convince them to apologize and stop. If some continue to flout these rules when the comments come back, then please email us to nominate those who who deserve to be banned. However --and note this well -- please follow guidelines (borrowed from the DailyKos policies for their "Hide Ratings"):

• Do not request that we ban someone for expressing contrary opinions, so long as they do so in a civilized fashion.

• Do not request that we ban someone you are actively having a fight with.

• Please understand that we won't have time to respond personally to email received this week or to resolve disputes, we decide what we consider offensive and all decisions are final.

By Mark Blumenthal on March 17, 2008 6:58 PM | Permalink | Comments (12)

March 15, 2008

Our Comment Bugs and Typekey Sign-in

By Mark Blumenthal

We want to update regular readers on two important issues affecting our comments facility.

First, we believe we have, at long last, identified and eliminated the technical glitch that caused long delays and error messages for those posting comments, frequently resulting in inadvertent double and triple posts. As of last night, hopefully, comments post without error messages within 10 to 15 seconds of clicking the "post" button.

I know our various comment bugs have been a source of frustration for just about everyone that has tried to post a comment on Pollster in recent months, and I want to once again apologize for the inconvenience we caused. Hopefully your experience will be far smoother from now on. If not, and in particular if you encounter any error message while commenting, please do not hesitate to email us with full details. We are committed to squashing these bugs once and for all.

Please note that it may take a minute or so for your comment to appear after you post it. To check that your comment has posted successfully, wait a full 60 seconds and click your browsers "refresh" or "reload" button. If you are really impatient, you can also perform a "hard reload" of the page (details here).

Second, now that comments are posting as they are designed to, we want to address another nagging problem resulting from the dramatic increase in traffic over the last few months. On Monday, if all goes well, we will begin requiring a valid Typekey username in order to post on Pollster.com.

Typekey is a free, third-party system that allows you to create a unique username. The registration is fairly painless -- you just need to be a real person (not a SPAM "bot") and leave a working email address with Typekey that can be used to validate your account. However, when you post a comment with a Typekey username, we will not be privy to your personal information, so you can remain completely anonymous.

We are making this change in response to your requests for greater moderation of the increasingly uncivil posts in our comments section in recent weeks. This change will hopefully discourage some of the worst behavior and put us in a position to band the most abusive commenters. It is not a panacea, of course, but it's a start.

If you do not have one already, I recommend that regular commenters go ahead and acquire your desired user name now. Please note that we may hold off on implementing Typekey sign-in if the new procedure creates any new delays or error messages like the ones we just worked to eliminate. We will keep you posted.

Finally, we are committed to maintaining an open comments section that allows for the free and civil exchange of views and dissenting opinion. If you have thoughts about how we might improve the tone of the discussion, please leave a comment below.

By Mark Blumenthal on March 15, 2008 11:27 AM | Permalink | Comments (13)

February 16, 2008

DC-AAPOR event on Wednesday

By Mark Blumenthal

An announcement for those in the DC area. I will be participating in a discussion on "Politics and Polling" next Wednesday afternoon hosted by the DC Chapter of the American Association for Public Opinion Research (DC-AAPOR). Hope you can join us.

By Mark Blumenthal on February 16, 2008 9:22 AM | Permalink | Comments (1)

February 15, 2008

Bialik on Poll Mash-Ups

By Mark Blumenthal

Carl Bialik, author of "The Numbers Guy" column for the Wall Street Journal, takes a balanced look today at the pitfalls of something we do here at Pollster, "mashing up surveys from various sources this election year to produce composite numbers meant to smooth out aberrant results." His piece is worth reading in full, as it considers both the benefits and risks of creating composite trends or averages:

Stirring disparate pollsters in one pot has its critics. "That's dangerous." says Michael Traugott, professor at the University of Michigan, and author of a recent guide to election polls. "I don't believe in this technique."

Among the pitfalls: Polls have different sample sizes, yet in the composite, those with more respondents are weighted the same. They are fielded at different times, some before respondents have absorbed the results from other states' primaries. They cover different populations, especially during primaries when turnout is traditionally lower. It's expensive to reach the target number of likely voters, so some pollsters apply looser screens. Also, pollsters apply different weights to adjust for voters they've missed. And wording of questions can differ, which makes it especially tricky to count undecided voters. Even identifying these differences isn't easy, as some of the included polls aren't adequately footnoted.

Bialik quotes both me and Charles Franklin in the column, but here are a few additional thoughts. We do not consider the trend estimates to stand as worthy replacements to the data from individual surveys. The trend lines -- and the estimates derived from their end-points -- are best considered as tools to help make sense of the barrage of often conflicting results from individual surveys. We learned in 2006 that "mashing up" surveys and "smoothing out" the variation between them helps counter the instinct to overreact to variation between individual polls -- some of it clearly aberrant -- that is common in hotly competitive political races. Moreover, while we only plot a few summary measures here such as vote preference and job approval, many of the surveys we report and link to include a wide variety of questions that help illuminate many aspects of public opinion.

Bialik is correct to argue that benefits of averaging lessen when we start to see large and consistent "house effects" separating the results from different pollsters. If a few polls are providing good estimates, while many other polls have misleading results, the mashed up averages may reflect more of the bad than the good. I wrote as much just before the Iowa Caucuses. Bialik correctly notes that the averages were misleading in California, where most polls showed the Clinton-Obama race closer than it turned out to be. His suggestion that we could "bolster" the case for trend estimates or averaging by comparing those numbers "directly against those from individual polling firms in terms of election accuracy" is a good one and something we are working on.

Bialik adds some additional detail in a companion blog item that focuses, among other things, on my calls for greater disclosure of methodological details, which includes a response of sorts from Zogby International:

When I asked Zogby spokesman Fritz Wenzel for further details, such as what those flawed estimates were, and passed along a blog post from Mr. Blumenthal calling for more disclosure from the firm, Mr. Wenzel dismissed sites like Pollster.com as “rivals.” “We are satisfied that we have identified the problem in California,” Mr. Wenzel wrote in an email, “and giving our rivals more ammo in the form of methodological detail, some of which is proprietary, with which to criticize us further doesn’t make the world a better place.”

Bialik is asking his readers comment on the value of composite poll numbers and whether better disclosure would "make the world a better place. Your comments are welcome here or there (or both!).

By Mark Blumenthal on February 15, 2008 12:03 PM | Permalink | Comments (9)

January 30, 2008

February 5 Update

By Mark Blumenthal

For those who may have missed it (I tucked a mini-announcement in last night's exit poll thread), we are now reporting poll results for twelve thirteen of the February 5 primary states, and will be adding another half dozen or so in the next 24 hours for which only a handful of polls are currently available. The links below also appear in the right column throughout Pollster.com:

AL Dem, Rep
AZ Dem, Rep
CA Dem, Rep
CO Dem, Rep
CT Dem, Rep
GA Dem, Rep
IL Dem, Rep
MA Dem, Rep
MO Dem, Rep
NJ Dem, Rep
NY Dem, Rep
OK Dem, Rep
TN Dem, Rep

Needless to say, we will be hard at work updating these pages as new polls become available over the next week. If you know of a poll in the public domain, or if you spot a typographical error that we have missed in our haste to get these data posted, please email us (at questions at pollster dot com).

One important note: We only plot our regression trend lines on the charts when eight or more polls are available. When you do not see trend lines plotted, the estimate for each candidate that appears in the chart legend is the median result among all available polls.

In several cases, that median result is probably a less accurate estimate of the current state of the race than the most recent poll released. Consider Colorado. The median result shows Clinton leading by eleven points (34% to 23%), but four of the five polls were conducted more than four months ago. The most recent Denver Post/Mason Dixon survey, however, shows a close race, with 34% for Obama and 32% for Clinton.

So in looking at polls for February 5, if our trend estimate does not plot, we recommend focusing more on the most recent polls conducted in January 2008 than the median value we report in the chart legend.

Readers have also inquired about sites that tally delegate counts and provide more information about the number of delegates up for grabs in each February 5 state and the rules for their allotment. Tracking that information is well beyond the scope of Pollster.com, but we are happy to recommend two excellent resources put up by our partners:

• The National Journal has a unique Campaign Tracker page that lists all of the February 5 states and provides the number of delegates at stake, the type of election (primary or caucus), the rules for participation (open or closed) and the all important rules of delegate allotment. The page is free to non-subscribers.

• Slate's Election Scorecard (which plots our trend estimates and includes daily analysis by Slate's Chadwick Matlin) also features a running tally of delegate totals won by each candidate.

We know that many other news organizations are tracking delegates, but we would appreciate your support for our partners at the National Journal and Slate.

By Mark Blumenthal on January 30, 2008 2:39 PM | Permalink | Comments (10)

January 10, 2008

Our New Partnership with National Journal Group

By Mark Blumenthal

This news has been a bit delayed between the holidays and the early primaries, but we are very excited to announce a new strategic partnership between Pollster.com and the National Journal Group, the publishers of the National Journal, CongressDaily, The Hotline, The Almanac of American Politics, and NationalJournal.com. Pollster.com will remain an independent website, just as it is now, but we will write a column on NationalJournal.com and provide content from Pollster.com to the various in-house polling resources within National Journal Group publications. In addition, Charles Franklin and I will write a column and provide features that will appear on NationalJournal.com.

What has us especially excited here is the opportunity to work with the top notch journalists that are a part of the National Journal Group and it's parent, Atlantic Media, including my old friend Amy Walter and her colleagues at The Hotline Ron Brownstein of NationalJournal.com and Charlie Cook's Political Report. The move of our "world headquarters" that I hinted at was to the National Journal Group offices in Washington, which also puts us just a few floors away from our friends at The Atlantic, including Andrew Sullivan and Marc Ambinder.

My first column -- some thoughts about what the success of the Des Moines Register Poll says about the philosophy of behind likely voter models -- appeared earlier this week (and should still be free to non-subscribers).

And as long as we are in a moment of transition, I want to say a big public thank you to both Charles Franklin and Eric Dienstfrey, who of course, are very much involved in this new partnership. If you like the charts and graphics here at Pollster, all credit goes to Charles, especially since his contribution here comes on top of a demanding day job at the University of Wisconsin. Second, Eric Dienstfrey, the hardest working man in show business (or at least at Pollster.com), has been working more or less non-stop for the last few weeks updating our charts and tables and the poll updates on the blog. We are all a bit fried at this point and yes, we have missed a typo or two in recent weeks, but if you appreciate all this site has to offer, you might want to leave Eric and Charles a big thank you in the comments section below.

Finally, a huge continuing thank you to Doug Rivers and YouGov/Polimetrix, for their sponsorship and unwavering commitment to our editorial independence without which Pollster.com would not exist.

It is going to be a long, interesting year and we are looking forward to helping you make sense of polls and polling data the whole way through.

By Mark Blumenthal on January 10, 2008 3:50 PM | Permalink | Comments (11)

December 31, 2007

How to Plot Rolling Average Results?

By Mark Blumenthal

With today's update, we confront the technical issue of how to incorporate rolling-average tracking surveys into our trend charts. What follows is a quick summary of what rolling-average tracking is and how we will incorporate the results of these surveys into our trend estimates. A warning: This post is mostly technical. We just want to make sure we document our procedure for those who sweat such details.

Rolling-average surveys, like the Zogby tracking in Iowa, do a relatively small number of surveys each night but report the rolled-together average of the last several nights of calling (usually three or four). So yesterday's release was based on interviews conducted on Thursday, Friday and Saturday, while today's release is based on interviews conducted on Friday, Saturday and Sunday. We will save the discussion of the pros and cons of rolling-average tracking for another day (though readers are free, as always, to chime in with their comments).

In 2004, Zogby did rolling average tracking in Iowa and six different organizations ran rolling average tracking in New Hampshire. The various media/pollster partnerships have shifted somewhat since, but we assume that the first waves of other New Hampshire rolling average surveys will begin to appear soon.

When including rolling-average results in the data that produce our trend line estimates, we want to avoid including each day's numbers as the base of interviews largely overlaps and the frequency of releases means that rolling averages tend to dominate simpler one-sample surveys (which usually spread interviewing over multiple days). So we will update our charts in a way that ultimately plots only every third release (for a three-day rolling average), but includes the most recent release as the final point for that pollster.

Here is the procedure we will use to avoid including data based on overlapping samples. Suppose a pollster uses a three-day rolling average:

• On the first day, we will plot their first release on the chart.
• On the second day, we will plot their second release and temporarily remove the first day's data from the chart.
• On the third day day, we will plot their third release and remove the second day's data from the chart.
• On the fourth day, we will add back the first day's release (and it will remain permanently) and also plot the fourth day's release.
• Going forward, we will continue to plot only the most recent release for each three-day window, restoring every third release so the chart ultimately includes no overlapping samples.

The table of results that appears below the standard chart on each poll page will, however, include all poll releases (with links to source data) for every pollster. Those surveys not plotted on the chart will be marked with an asterisk (*).

By Mark Blumenthal on December 31, 2007 2:35 PM | Permalink | Comments (0)