February 28, 2008
Telling Good Polls From Bad
My latest column for NationalJournal.com, which discusses measuring a survey's merit, is now online.
By Mark Blumenthal on February 28, 2008 1:52 PM | Permalink
Comments
I assume that polling organizations have a fairly good handle on the uncertainties associated with their respective "likely voter" models. Disclosure of the uncertainty should be a primary requisite for a "good poll." Non-disclosure or (worse) ignorance of the uncertainty should be a prime indicator of a "bad Poll."
Just a quick thank you to Mark for taking the time to answer my question.
Just a quick thank you to Mark for taking the time to answer my question.
Excellent column. Newspaper and teevee stories on polls never, never go into the complexities of survey research to the extent you guys do here. It's all just Clinton 48%, Obama 43%, "statistical dead heats" and a margin of error + or - 4 points. There is almost never any comparison to other polls for context (esp. if the news outlet sponsored the poll), even though it can be found here in about 5 seconds.
You are doing exemplary work and those of us who tcome here to read your analysis and not just lob insults are learning a tremendous amount about polling.
There is a lot to comment about in the column. Here is one issue.
Re: or should we sample from registered voter lists that allow for a more accurate identification of actual voters but miss those with non-listed phones?
The quality of what we call RBS samples differ from state to state. In some states they are non-existent.
There are states that obtain voter phone numbers from the voters themselves when they register. So these do include unlisted numbers. Not only do they include unlisted, they include cell phone numbers too.
List brokers may also enhance these lists by doing phone number matching against phone number lists.
Unlisted numbers also vary from state-to-state. According to Survey Sampling, a provider of samples of all kinds, states with the highest percentage of listed phone numbers are Vermont 94%, Maine 89%, and (get this) New Hampshire with 83%.
Re: accurate identification of actual voters. Not evident here is the opportunity to pre-stratify samples by area according to past voting patterns so that each area of the state is properly represented. Not doing so would typically mean under-representing urban areas of a state with lower interview cooperation rates.
This is especially useful in newly redrawn post-Census congressional districts. This may have been the case when Blumenthal's firm showed Melissa Bean (D) winning the IL 10th in 2004 against 30+ year incumbent Phil Crane. A listed sample would insure you are interviewing voters who live in a district.
All for now.
Dear Mark,
I like this website much because it is non-bias. I am looking at some poll lately said Hillary lead among Hispanic supports is shrinking. But I just read the poll by Lanitos decision siad Hillary lead is expend with 62-21 among latino and hispanic vote. please comment
Posted on February 28, 2008 2:31 PM