Pollster.com

Articles and Analysis

 

Trackers and House Effects

Topics: House Effects

TrackerHFX.png
An update on the house effects of the daily trackers. Here I'm also showing the effects of RV, LV or LV1 or LV2 for Gallup. For Hotline, the shift from RV to LV made little difference. But here you can see considerable difference for Gallup's LV1 (attitude only) and LV2 (traditional method) of identifying likely voters.

All data here are through 10/29.

The daily trackers combined into a single estimate follows quite closely with out trend estimate that also includes stand alone polls, as seen below. Note that the standard trend estimate DOES include the independent samples of trackers, while the trackers only estimate is for every release of the trackers, including overlapping samples.
dailytrackers.png

 

Comments
NorthernObserver:

Hotline seems most consistent.

____________________

DaveVatDK:

Charles -- you should include a time covariate interaction in your model to capture trends, as house effects clearly decline over time as the polls converge. Importantly, they converge at different rates -- for example, R2k (Daily Kos) has shown more tightening than the other trackers, thus reducing its house effect over time. I would wager that the bigger the house effect, the larger the trend in which that house effect dissipates. Your analysis could lead to a perhaps mistaken conclusion that, moving forward from today, the polls with the largest house effects are the least accurate.

Also, note to everyone: the center line in this graph is not the "truth" -- it's the poll-of-polls average. So, these do not represent bias in any sense of the word, as we do not yet know the true population preferences.

____________________

mattn:

This is an interesting analysis, but I wonder if some house effects are bscured by not looking at the separate Obama/McCain numbers. That is, Dr. Franklin's analysis holds for the margin, but is it true for the individual numbers as well? Hotline, for example, is always below the Obama trend line, possibly even outside the confidence interval (I'm guessing).

____________________

DaveVatDK:

@NorthernObserver --

I think you may be misinterpreting the graph. Hotline's average is closest to the poll-of-polls average. The precision of the estimated house effect (width of the blue bar around the red dot) for Hotline is the same or larger than most other polls.

____________________

jme:

- Are you using the average sample size of each tracker for these CIs?

- Anyone else interested in how much variation Gallup's likely voter models are adding to their estimates? Hotline's LV model is also increasing variation. It's hard to tell from the graph, but when I squint it looks like ABC/WP's LV model is also (slightly) increasing the variability of their estimates.

- I know Daily.kos has been quite open about their LV model, and so has Gallup (I think), but I don't follow LV model stuff enough to know about the others. I'd love to see a side by side comparison of what each organization is doing. That might help us tease out which assumptions are influencing both the house effect and the level of variation.

____________________

DTM:

So we know that the tracking polls are largely coming out where expected in terms of house effects based on their partisan ID practices and likely voter models. But if I am not mistaken, Gallup and ABC/Post are also the only two trackers to call cell phones. Indeed, I might suggest that actually may be the chief reason that Rasmussen, who I do not believe calls cell phones, comes in a bit lower than those trackers.

So, I'd actually be willing to go out on a limb and suggest a "best practices" tracker average (one assuming all the polls made a few reasonable adjustments to the likely voter model and partisan ID numbers and included cell-phone-only-users) would be a point or two higher than the current average. Not a big deal, perhaps, but I thought it was worth noting.


____________________

coltraning:

A couple of odd things indeed...Why does gallup LV2 show a republican lean, when it has consistently shown a bigger Obama lead than LV 1? Is that a mistake. Also, it is interesting that virtually all of the stand-alones show a larger Obama lead than the trackers, so how is the "norm" for trackers determined?

I also think the point about cellphones is an excellent one, as that would explain much if not all of the gap between the various trackers. Is there anyway to run the numbers through a screen where the same demographics are applied to all the polls. Say use a plus 6 Dem ID, same likely voter models...I guess we will know on Tuesday (barring massive vote shenanigans) whose LV model is right, and whether there was indeed a cell phone effect. My guess? 52-46-2

____________________

drdr:

Thanks for this analysis, Mr. Franklin.

Today Obama is +8 on Gallup LV1, which the chart shows to have minimal partisan lean.

Interestingly, Obama is +11 with RV on Gallup, the highest he has ever been.

Gallup shows no signs of the race tightening.
I believe Gallup includes cellphones in their survey.

____________________

UpstateProgressive:

I too am confused by the Republican lean of Gallup's LV2 model. I'm wondering if the LV2 and LV1 numbers have been flipped?

This is going to be an interesting test of the pollsters, and the smart money is probably on a composite of the different models. Interestingly, Rasmussen (often accused of leaning right) and Hotline (new to me this year) are nearest that average.

However, my gut (and the early polling results) tell me this election will be a wave election bringing scads of new voters coming to the polls, and in that case the Dem leaning polls with their more inclusive LV models will prove to be more right. Either way, Obama should win, but if the Dem leaning polls are right, it could be a true tsunami.

In any case, I'll be very surprised if IBD/TIPP proves to be the most accurate pollster of all.

____________________

jme:

LV models would appear to generally reduce Obama's support, but Hotline's has little effect. But would Daily.Kos's RV numbers really be even further the right? I doubt it.

Are IDP/TIPP, GWBattleground and GallupLV2 similar just by chance or are they doing very similar things?

Why is ABC/WP so variable, even compared to polls with similar sample size?

Why is Gallup so variable in their LV models despite having the second highest sample size?

So many questions...

____________________

Sorry to be slow replying-- I was on the road and had limited net access.

coltraning: The Gallup LV1 is their "expanded" model while LV2 is the "traditional" model. Traditional has shown a consistently closer race than has the expanded or RV. So I'm pretty sure this isn't mixed up. Maybe LV1 and LV2 aren't universally understood as to which is which. (UpstateProgressive too-- do we have different understanding of what LV1 and LV2 are? Can you point me to a "Official Gallup" usage?)

jme: the effect CI is based on the coefficient standard error estimate, not the average sample size (as in our classification method.)

Also note that the CI is a function of the number of polls as well as variability, so beware comparing CIs because they reflect differences in when each organization started the tracker or how long they collected RV or LV (as in Hotline).

And jme again: the sample sizes of the surveys plays a minor role (not none, but not huge). It is the number of polls that has more to do with the precision, plus the variability of the tracker results.

And to several: I wish I had better information about what exactly each pollster is doing for LV. I don't.

Thanks all.

Charles

____________________

jme:

Aha...thanks for the clarification, Charles!

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR