Pollster.com

Articles and Analysis

 

Bialik on Nielsen's New College Numbers


Carl Bialik, the "Numbers Guy" at the Wall Street Journal, devotes his column this week (free to non-subscribers) to a change in the Nielsen television ratings to include college students living in dorms. Not everyone realizes that the Nielsen ratings are based on a random sample survey - a randomly selected panel of households that agrees to let Nielsen monitor its viewing habits. According to Bialik, while the inclusion of roughly 2.1 million college dorm residents into the Nielsen sample has increased the audience size estimate for some shows among 18-24 year olds by as much as 60%, it is based on a very small sample:

[T]o calculate these estimates, Nielsen is extrapolating from the viewing habits of just 130 students around the country who have agreed to have electronic monitors installed in their dorm rooms. That means the decisions of a handful of those students can lead to a huge swing in ratings -- that 163,000 jump for "Drawn Together" was based on 12 people in Nielsen's survey group who tuned in to the show.

Bialik's piece explains the mechanics of the Nielsen ratings and goes into depth on the question of whether that sample of 130 is large enough. A Nielsen spokesman says the sample is "adequate, but some academic statisticians Bialik talked to had doubts.

Statisticians I've spoken with told me national surveys typically include at least 500 people, to limit the margin of error. (An ongoing Princeton University survey of college students, examining minority students' academic progress, tracked about 4,000 students.) Political pollsters often survey just 1,000 or so people to determine broad national trends, but sampling error is less of a problem when respondents are choosing among two presidential candidates. It's a different matter when asking someone which of 100 or more channels they are watching. Mr. Holmes said Nielsen isn't calculating a margin of error for the college group.

News reports and promotional press releases usually don't make it clear how small samples can influence ratings. They sometimes crown networks, shows and personalities as winners based on differences of just 0.1 rating point (representing about 110,000 households). "The ratings just aren't accurate to that point," said Lynne S. Gross, professor of radio, TV and film at California State University, Fullerton. Nielsen's Mr. Holmes said the company supplies information to clients that would allow them to calculate an overall margin of error that they could publicize.

It's a fascinating column. Go read the whole thing.

 

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR