Pollster.com

Articles and Analysis

 

Day-of-Week Effect in Gallup Daily?

Topics: 2008 , Barack Obama , Ben Margolis , Harrison Hickman , John Edwards , Rasmussen , Sampling

I received some interesting charts this afternoon from Harrison Hickman and Ben Margolis, both of the Democratic firm Global Strategy Group that polled for John Edwards until he withdrew from the race (full disclosure: Hickman was my employer longer ago than either of us wants to admit). When Hickman and Margolis plotted the Gallup Daily results for the Obama-Clinton race, they noticed an impressively consistent pattern by day of the week. In data released during February and March, Obama typically does best on three-day samples that end on Saturday (combining interviews from Thursday, Friday and Saturday), while Clinton typically does best on samples ending on Tuesday or Wednesday night (covering Sunday through Wednesday).

The chart below shows the pattern in the Clinton lead (Clinton minus Obama) for each of the last nine weeks:

Hickman and Margolis sent a second chart which shows the average for each daily release across the nine-week period. On average, the results have shown a more than three point shift from Sunday-Monday-Tuesday (Clinton ahead by 1.3) to Thursday-Friday-Saturday (Obama ahead by 2.6).

03-31dayofweek_avg.png

So, given this pattern, do not be surprised if the ten-point Obama lead reported yesterday (based on a interviews from Thursday through Saturday) narrows a bit more by mid-week from the eight -point lead reported today. The results also suggest that we would get a less "volatile" sense of the race by looking at a seven-day rolling average that eliminates the apparent day-of-week effect.

The more interesting question is, why are we seeing this pattern? It seems counterintuitive, at least at first blush. I would have expected Obama's supporters to be harder to reach on weekends, given that they tend to be younger than Clinton supporters. Back in December, Obama's supporters seemed to be harder to reach around Thanksgiving and the Christmas holiday. Unless I am missing something obvious, this pattern is different.

Any thoughts? Theories?

UPDATE: Hold the, er, phone. I sent Hickman and Margolis a spreadsheet with the Rasmussen Reports daily tracking and they generated comparable charts. Oddly enough, the pattern is almost the mirror opposite of Gallup. On the Rasmussen automated tracking (which reports a four-day rolling averge), Obama does consistently better in the middle of the week, Clinton better on the weekends. Here is the chart of each week:

And here are the average values for each daily release:

I will have a chance to add more thoughts later, but I think the bottom line is that the conflict of the patterns shown by the two pollsters suggests that the apparent day-of-week effects are about differences in poll methodology, not a real variation in voter preferences over the course of the week.

UPDATE 2: Both in comments and in email, readers have questioned the statistical significance of the patterns in the charts above. I am persuaded that they have a point.

One argument many are making, with good cause, is that the values for adjacent days are not statistically independent. In other words, the value plotted for each day for Gallup represents a rolling average of three days of data (and for Rasmussen, four days). So it is not surprising to see smooth day-to-day trends. That's the point of the rolling average, something I've observed previously.

We can, however, isolate two independent measurements each week for Gallup from the available data. For example, there is no overlap between the data collected on Sunday, Monday and Tuesday (labeled "Tuesday" on the chart above) and on Thursday, Friday and Saturday (labeled "Saturday). Looking at the first chart above, we see what certainly looks like a "consistent" pattern: In seven out of eight weeks, Clinton does at least a point better in the Sunday-Monday-Tuesday sample than than the Thursday-Friday-Saturday sample.

If random chance were the only factor at work, then the odds of seeing a difference one way or another between the two independent samples in any given week should be like flipping a coin. The odds of flipping a coin and having it come up the same way seven times (either heads or tails) is about 7% -- highly improbable but still slightly higher than 5%, the level of confidence we usually require to call something "statistically significant."

However, as my friend Mark Lindeman tells me, my test is sketchy in several ways. Tuesday was primary day in many of these weeks. The biggest drop on the chart, for example, occurs during the first week of February following Super Tuesday. If we throw out just that one week, then odds of the late week drop occurring by chance alone on the Gallup series increases to about 13%.

Moreover, if we look at all eight weeks, but compare the Clinton margin each Sunday-Monday-Tuesday sample to the preceding Thursday-Friday-Saturday, the consistency seen in the chart disappears: Four ups, two downs and two comparisons that show no change. The odds of getting that pattern by chance alone are not much better than 50-50.

Things are a bit more complicated with the Rasmussen data, since the four-day rolling average makes it impossible for us to isolate two independent measurements each week, but the pattern illustrated in the chart is less consistent by week than for Gallup.

Others may suggest better approaches to the statistics and, of course, patterns in future weeks will add further evidence one way or the other. However, I am convinced that these patterns owe more to random chance than I had first assumed. Hopefully, the analysts at Gallup and Rasmussen Reports, who can run tests using the independent daily samples, will help clarify.

One more thing: I should be clear that while the charts above came from Hickman and Margolis, the commentary and conclusions were mine and mine alone. So blame me for any confusion.

Update 3: Gallup's Jeff Jones responds with data since January broken out by day-of-week.

Update 4: Harrison Hickman adds his thoughts as well.

 

Comments
Tom Veil:

Could it be that the news cycle has gotten that regular? In other words, perhaps this is not a flaw in the polls, but rather that pro-Obama/anti-Hilary stories are systematically coming out around Thursday while pro-Hilary/anti-Obama stories are systematically coming out around Sunday? I know it sounds odd, but since TV network programmers make their money off of the unconscious urge that we all have to, say, watch game shows on Wednesdays sitcoms on Thursdays, perhaps it's not too far-fetched that journalists and opinion-makers are unconciously doing something analogous.

____________________

lemonfair:

During some of the last month Hillary was getting very positive coverage on Saturday Night Live, which might affect the next few days.

____________________

I like the SNL idea, but I wonder if the Sunday papers and/or talk shows have been especially favorable to Clinton.

I do agree that if there is any real effect, it's gotta be media-driven. Unless people get into that "Hope" thang more at the end of the week. :-)

____________________

bobbyford:

Could this be a result of Obama's higher support among those who are likely to work regular office hours? Obama does well among groups who are likely to be away from home in the week: the highly educated, men, and professionals.

Hillary support is strong among groups more likely to be at home during the 9 to 5: women, pensioners and the working classes, who are more likely to be working unconventional hours.

Hillary's supporters may therefore be easier to get hold of during the working week. Obama's college kid supporters may be hard to reach at weekends, but they are only one part of his coalition (albeit a vocal one). The white collar professional men who also back him by big margins are likely to be much easier to catch when relaxing at the end of a long week at the office.


____________________

Jeff:

Maybe Obama got a bump from a lot of the primaries he's won the past couple months, many of which fell on Tuesdays.

____________________

lrhirshman:

I can't help wondering if the blue collar women, who are her main support, are at home to be polled on those Sundays and holidays. I would normally assume that Gallup would correct for all these demographic factors, but the gendered composition of the household/second shift stuff is newly important this election cycle, and I wonder if they have fully corrected for it.

____________________

Brackdurf:

Standard deviations?

Just eyeballing the graphs, I estimate a SD of about 6 for the blue peak and the green trough. So, probably no significant difference either between the two, or over time in either.

____________________

Mercan:

There is little reason to suspect that these data exhibit a weekday effect. Random data will exhibit patterns when you look for patterns. There are 5 observations that upshift on Tuesday and there are (less visibly) 3 observations that downshift on Tuesday, this is 1 observation away from 50:50, what is non-random about that?

Six Monday observations upshift, one Sunday observation is missing so upshift evaluation is not possible, and one downshifts, at least there is a case for non-randomness, but with 7 observations and one period, how can you tell?

It is 3 to 5 on Thursday and 3 to 4 on Friday, there is simply NO EVIDENCE of non-random variation.

____________________

Mercan:

Oh, by the way, the big downshift on Wednesday in the Rasmussen data is equally unsupportable. There are 4 ups and 4 downs, the data are completely random.

____________________

kingsbridge77:

Is this effect perceived in the GE matchups as well?

____________________

Mercan:

By the way (again), it is a safe bet that Gallup will "narrow a bit" towards mid-week. This phenomenon is normally called regression to the mean.

____________________

lemonfair:

the_real_truth:
Thanks for the link. Have been waiting for a shift in PA for quite a while, but this site has had the 51-39 split for a very long time.

____________________



Post a comment




Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.

MAP - US, AL, AK, AZ, AR, CA, CO, CT, DE, FL, GA, HI, ID, IL, IN, IA, KS, KY, LA, ME, MD, MA, MI, MN, MS, MO, MT, NE, NV, NH, NJ, NM, NY, NC, ND, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VT, VA, WA, WV, WI, WY, PR