Pollster.com

Pollster.com: Polls and Charts

 

2008 National Presidential Primary Sensitivity Comparisons

ATopDemsSensitivity4Up600.png

Regular readers know that my "standard" trend estimate is the blue line in the charts. This is a line that is calculated to go through the "middle" of the data, with an average error of zero, meaning the points below the line balance the points above the line. "Old Blue", as I affectionately call this line, is deliberately conservative in the sense that it takes quite a bit of new polling data to convince it to change trend direction. The reason for this is that we know there is quite a bit of noise in the polls (just look at the spread of points around the line!) so when a new poll comes in high it might mean an upturn in support, but it is just as likely that it simply reflects random noise and the next poll is as likely to come in low. If we allow the trend estimate to chase each new data point too much, we'll just plot random noise rather than the best estimate of the trend in support. Experience with these and other data (such as presidential approval) has shown that Old Blue is seldom misled about new trends, though it does take a while (about a dozen polls) to notice changing trends.

While it is good to avoid responding too much to a single poll, it is also true that Old Blue may stick to a trend longer than it should. A more sensitive estimator would notice a change in direction quicker, and would jump on the new trend while it is still news-- and before others notice it. "Ready Red" is the answer to this. The red line in the charts is twice as sensitive to change as is Old Blue. As a result it will pick up changes in momentum more quickly, letting us spot new trends early. Unfortunately, it will also sometimes be misled and will think it sees a new trend when in fact none exists-- just a few polls that happen to be "down" or "up" but which really don't represent any significant shift.

Of course you can adjust the sensitivity of the trend estimator to anything between Ready Red and Old Blue (or outside them too, for that matter) to see how much difference the sensitivity makes. There is no perfect way to choose a "best" estimator. I've settled on the more conservative Blue estimator as my standard because I find the hasty red estimator has often jumped the gun on presidential approval trends, which more data has subsequently shown were not really changing. But that doesn't mean we shouldn't examine the more sensitive trend estimate-- it tells us a lot, even if we have to be a bit cautious. (I'd rather be right but slow. Others prefer to be quick, and adjust to mistakes as necessary. Both approaches have their virtues.)

The chart above shows the data and the trend estimates for the top Democrats. Republicans are in the chart below. In addition to Old Blue and Ready Red, there are a number of gray trend lines. These show the estimated trends for levels of sensitivity from quite a bit MORE sensitive than Ready Red to MORE conservative than Old Blue. If you can see the gray lines, this means that at least for some levels of sensitivity the estimated trend differs from either Blue or Red. If you cannot see the gray lines, or only barely, this means that the estimated trend hardly depends on the amount of sensitivity and the many gray lines all lie under Red or Blue in the plot, and so are covered up. This typically happens when there is a smooth, steady trend with no bends in it.

ATopRepsSensitivity4Up600.png

We can check the sensitivity of these estimates to the amount of smoothing used to estimate the trend. Here I use separate estimates of the current standing of each candidate, with the smoothing ranging from MORE sensitive than Red to MORE conservative than Blue. This is a wider range than I think anyone would reasonably want. The most sensitive end produces trends that jump around way more than anyone could believe, and the most conservative fit is basically just a straight line with hardly any change at all. But somewhere between these limits of silliness are a range of reasonable estimates. If the bottom line estimate for a candidate is pretty compact, then the amount of smoothing doesn't matter. If the estimates are spread out, then we at least know that sensitivity matters and we should be cautious. The summary of the data are presented below.

ADistributionofEstimates600.png

One final way to look at sensitivity is to plot the estimated support for each candidate against the degree of smoothing used for each of the estimates. Low values are less smoothing and more sensitive trends, while high degree of smoothing are more conservative and less sensitive.

Sensitivitytosmoothing.png

It is good to compare Old Blue and Ready Red-- both offer helpful insights into the nomination race. Your acceptance of the risk of being too slow to recognize change versus the risk of chasing phantom blips should help you decide which to give more credence.

- Charles Franklin

View all National charts and poll data here.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR