December 31, 2007

Three Estimates (Explained) for IA & NH

As of today, our Iowa and New Hampshire charts pages now include two new features: A summary of the current estimates for each candidate based on the trend lines on our charts (including the frequently requested "sensitive" estimator), and a regularly updating set of "sensitivity" analysis charts that compare each trend line or all candidates.

Charles Franklin posted an in-depth analysis on December 21 that walks through the rationale for these three trend estimates and their meaning in greater detail, but here is a quick capsule summary:

• Standard Estimate (blue) -- The charts at the top of each page reflect our standard estimate, a local-regression based trend line. The standard estimate is deliberately set to be conservative in that it takes a good bit of evidence of "real" change (i.e. more than one or two contrary polls) before the trend will show a sharp turn. With lots of polling, this estimator has an excellent track record of finding real turning points of opinion while not chasing the "wild geese" of opinion polling.

• Sensitive Estimate (red) -- Appears in the "Sensitivity Analysis" charts toward the bottom of each page. The sensitive estimator uses the same "local regression" methodology as our standard approach, but sets the degree of smoothing to about half that of the standard blue estimator. The sensitive estimator should detect short term change more quickly than "blue", but it will also chase phantom changes due to flukes of a few polls that happen to be too high or too low.

• Last-5-Poll Average (black) - Also appears in the "Sensitivity Analysis" charts toward the bottom of each page. Unlike the regression based trends, this estimate draws only on data from the five most recent polls. It shows more short term variation due to the impact of the "house effects" in polling organization practices.

Our experience is that each of the estimates tend to move in rough agreement with each other, typically showing differences in the range or a percentage point or two of each other. Polls are generally not able to estimate such small differences in candidate support, but feel free to make your own call based on how well the trend lines "fit" the available data.

Remember that our charts plot the trial-heat vote results on all public polls released in each state. The trend lines represent estimates based on a combination of all available data.

Readers frequently ask how our estimates differ from the averages that appear on RealClearPolitics. Their method comes closest to the last-5-poll average, although they vary the number of polls in their average on any given day depending on the timing and frequency of recent surveys. Vive la difference!

Readers also ask about how regression trends differ from simple averaging. Here is Franklin's answer, from a recent post:

Our trend estimate is just that, an estimate of the trends and where the race stands as of the latest data available. It is NOT a simple average of recent polling but a "local regression" estimate of support as of the most recent poll. So if you are trying to [calculate] our trend estimates from just averaging the recent polls, you won't succeed.

Here is a way to think about this: suppose the last 5 polls in a race are 25, 27, 29, 31 and 33. Which is a better estimate of where the race stands today? 29 (the mean) or 33 (the local trend)? Since support has risen by 2 points in each successive poll, our estimator will say the trend is currently 33%, not the 29% the polls averaged over the past 2 or 3 weeks during which the last 5 polls were taken. Of course real data are more noisy than my example, so we have to fit the trend in a more complicated way than the example, but the logic is the same. Our trend estimates are local regression predictions, not simple averaging. If the data have been flat for a while, the trend and the mean will be quite close to each other. But if the polls are moving consistently either up or down, the trend estimate will be a better estimate of opinion as of today while the simple average will be an estimate of where the race was some 3 polls ago (for a 5 poll average-- longer ago as more polls are included in the average.) And that's why we estimate the trends the way we do.

-- Mark Blumenthal

December 31, 2007 in Pollster.com, The 2008 Race

Comments

Post a comment

URL:

Comments: (you may use HTML tags for style)

Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button.

Please email us to report offensive comments.

See our comment policy here. Note that we require commenters to share their email address via Typekey. We will never share your email address with anyone without your explicit permission.