Articles and Analysis


Anthony Wells Interview: Part 2

Topics: Internet Polls , Interpreting polls , Measurement , Sampling , UK elections

Anthony Wells is the editor of the UK Polling Report and an associate director at YouGov [interests disclosed: YouGov is Pollster.com's parent company]. He spoke with Emily Swanson on Tuesday about polling and the UK elections. Below is part 2 on polling methodology. Part 1 on the state of the race and interpreting polling data is available here.

Could you tell me about some of the different pollsters active in the UK?

We've got about 5 who I'd call established pollsters. Just in the run-up on this campaign there's been a lot of new entrants. But over time, there's been about 5 since have been there since the last election. Four of those are telephone pollsters, so they'll all use random digit dialing, and IPSOS-MORI has some degree of quota sampling in terms of who they asked depending on who in the household picks up the phone. The other three I think are pretty much random. The fifth one is ours, YouGov, and we have a panel-based internet methodology, so our sampling basically is quota sampling.

You said [in Part 1] that up until last week it looked like a Conservative blowout and suddenly it doesn't really look like that anymore. Given how quickly things can change, is it difficult for pollsters and analysts to deal with the brevity of the election cycle?

It's not really a problem in that sense - what I always noticed, the difference between US polling and UK polling is that away from election times the main currency of US polling seems to be presidential approval rating. And the generic, would you vote for Republican or Democratic candidates in congressional elections, is pretty much divided. Here, the currency is voting intention, because we always know who the alternative prime minister is going to be. So that question is asked, year in, year out, throughout the whole parliamentary term. So really, the sort of questions that pollsters ask in an election campaign are much the same as the ones we ask outside an election campaign, we just ask it more often.

So it's much like our generic congressional ballot, where they'll ask throughout the entire cycle, 'if the election were held today...'

Yeah. Like that, but we pay it far more attention. We don't really pay much attention to government approval rating, where presidential approval rating seems to be the question that everyone looks at in the US.

It seems like YouGov has really caught on in the UK, and generally speaking there's more acceptance of the idea of internet polling.

Well it was a long time, and basically we kept getting things right. There was extreme distrust to start with, and then we got the 2001 election right, and then got several sort of "mid-term" elections right, and got the 2005 election right as well, and after that point I think we began to be accepted. A stopped clock tells the right time twice a day, but if it keeps telling the right time you have to basically concede that it's working.

It was sort of a hard slog to begin with. I mean now, I talked about the 5 main established pollsters - in the run-up to this campaign, there's been at least 3 or 4 new entrants who are largely online-based ones, basically following in YouGov's footsteps. At least 2 of them were founded by people who used to work for YouGov, and they're trying to go off and do it themselves.

Are a lot of the media outlets reporting on the newer internet polls, or are they waiting until those prove themselves as well?

In terms of newspaper media, they'll mostly report on polls they've commissioned themselves, however shoddy, and then they'll probably more often report the established pollsters if they're going to mention someone else's polling, but they do tend to be very parochial about it and make a big fuss about their own one, and mention other peoples' at the bottom of page 57. The broadcast media - largely the BBC - because we've got a much more limited pool of pollsters, they largely seem to do it on a case-by-case basis. In the previous election, they really were very sniffy about internet polling, and they mentioned the 4 main phone and face-to-face pollsters at the time and didn't mention internet as much. These days, we seem to be one of the ones they do refer to, while they'll ignore most of the new entrants, so it's all whether they're established or not, not methodology, in terms of the BBC.\

Is there anything else you think US audiences should know about UK polling or the UK elections more generally?

Actually there is something that might be worth pointing out as a difference, which is our figures nearly always exclude don't knows. We percentage them out, and US ones don't. Most companies just sort of ignore them. They just assume they won't vote of they'll vote in exactly the same way. Two of them, ICM and Populus, reallocate 50% of them based on what they voted for last time.

The common parlance is that it's the "Shy Tory adjustment" - they first started doing it after 1992, when the polls got it horribly wrong, and they underestimated the Conservative vote, and one of the reasons amongst others they thought was that people were embarrassed to admit to pollsters that they were actually going to vote Conservative. And they saw, looking at the people who were saying "Don't know," a disproportionately large proportion of those were people who had voted Conservative in 1987. There was also solid evidence based on post-election callback surveys that people who said don't know did tend to vote for the party they had done previously. So they reallocate 50% or so according to their previous vote. But, while people still call it the Shy Tory adjustment sometimes, it doesn't actually help the Conservatives anymore. Now it tends to help Labour - they're actually shy Labour voters now.

Are there any resources that American audiences should know about if they're interested in UK elections?

The main one's really are the BBC website would probably be the best place to start, and beyond that it tends to be the main newspaper websites, so the Telegraph, the Guardian, the Times, and so on.



This is an interesting interview, especially the nugget that British pollsters repercentage vote choices and eliminate undecideds.
What does Mr. Wells think about AAPOR's report on online panels - http://tinyurl.com/ylnyrqo
- of which the chief finding was:
"Researchers should avoid nonprobability online panels when one of the research objectives is to accurately estimate population values. There currently is no generally accepted theoretical basis from which to claim that survey results using samples from nonprobability online panels are projectable to the general population. Thus, claims of ‚Äúrepresentativeness‚ÄĚ should be avoided when using these sample sources."


Post a comment

Please be patient while your comment posts - sometimes it takes a minute or two. To check your comment, please wait 60 seconds and click your browser's refresh button. Note that comments with three or more hyperlinks will be held for approval.