Pollster.com

Articles and Analysis

 

Luntz: Uses Repeat Participants by Design

Topics: Focus Groups

Earlier today, TPM's Paul Kiel asked Frank Luntz about a participant in his most recent post-debate focus group for Fox News that had also appeared in prior televised focus group:

[W]hen I spoke to Luntz today, he said that he uses repeat participants by design. In a segment to air on Fox News tonight, he said, there should be a "bunch of people" who had been in prior focus groups, some of them participating as early as May of last year. "It allows me to see how people's opinion have changed over time," he explained. "I'm trying to isolate that moment that made the difference."

Before Kiel spoke to Luntz, he called me for a reaction. My own view, quoted in Kiel's article, is that the presence of repeat participants is a troubling indicator about of recruitment quality. But then, I had never heard of this particular application of what Luntz goes on to describe as "anthropology." Draw your own conclusions, I suppose, but the survey researchers I know try to avoid repeat focus group participants.

I wrote a brief introduction to political focus groups, their strengths and weaknesses, back in 2006. Then as now, I also highly recommend this story that aired on NPR's Marketplace in March 2002 on what can go wrong in focus group recruitment (the focus group story begins at about 4:10).

Update - Reader Mike G. leaves this helpful comment:

As someone who worked in market research for four years, I can tell you that it is viable to use the same group. This is what's known as a longitudinal method. However, Frank Luntz should have been up front about this at the beginning of the segment. The entire segment was presented like these voters had not been interviewed previously. This is intellectually dishonest and, in polling, such intellectual dishonesty has no place.

 

Comments
Jeff Winchell:

Looking at the Fox video of this (linked to from the TPM website) the people in that survey gave such snappy answers, with no delay after Frank would ask them a question. Every other focus group I've seen after one of these debates it looked more like herding cats. This one looked like herding paid soldiers. This thing smells VERY phony to me.

____________________

Brian:

Frank Luntz' focus groups have been shown to be nothing more than propaganda events. Makes me wonder if we should even be trusting Fox's "scientific polls".

____________________

donjo:

Fox "debate" and Luntz killed Wes Clark in 04 and are trying to knock off Huckabee in 08. To bad for Rush and Fox that Huckabee is leading nationally.

____________________

Asquared:

If Frank Luntz had huevos he'd let his focus
group focus on Fred Thompson's 17 minute position statement .......sheesh

____________________

"too bad for Rush and Fox that Huckabee is leading nationally." NO it is too bad for the country and the Republican Party.
This phony Elmer Gantry clone will only induce an independent candidate to run and cause many conservative voters to stay at home. Hey! the country survived Kennedy, Johnson,Nixon,Ford,Carter,Bush1,Clinton, and Bush2, we can "probably" survive Osama if he makes it.

____________________

As someone who worked in market research for four years, I can tell you that it is viable to use the same group. This is what's known as a longitudinal method. However, Frank Luntz should have been up front about this at the beginning of the segment. The entire segment was presented like these voters had not been interviewed previously. This is intellectually dishonest and, in polling, such intellectual dishonesty has no place.

____________________

John Galt:

Focus groups are anything BUT random...first you start with a recruitment list of people that have volunteered to be available for future groups, then you need someone that is able to spend 2-4 hours away from home at a specific time AND for the purposes of the group in question, you needed to be an undecided GOP primary voter (not to mention the simple fact that you needed to live within a reasonable distance from the focus group location).

Focus groups aren't random, they aren't scientific, and their results are anything but reliable. What's the big deal here???

Moreover, so long as the repeat respondent was truly undecided, I don't see a problem with that.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR