ARI SHAPIRO, HOST:
Something strange has been happening with polling. Ever since election night here, pollsters have been rethinking their work on the presidential race. As our colleague Robert Siegel reports, that season of self-scrutiny extends to other countries, too.
ROBERT SIEGEL, BYLINE: The French had a presidential primary in November - two rounds to pick a center-right candidate for president - there's an election this year - and needless to say, there was polling.
BRUNO JEANBART: My name is Bruno Jeanbart, and I'm deputy CEO at Opinion Way, which is a French polling company.
SIEGEL: A few days before the first round, Opinion Way's polling showed what most polls showed. Three candidates were in contention - former President Nicolas Sarkozy and two former prime ministers, Alain Juppe and Francois Fillon. Juppe looked like the leader.
JEANBART: We are Francois Fillon at 25 percent, Sarkozy at 25, and Juppe was 33.
SIEGEL: Eight points up on his nearest rival. But on primary night, it was the more conservative, more pro-Russian Francois Fillon in a walk. He had 44 percent of the vote - almost 20 points more than the poll had showed - and the following week...
(SOUNDBITE OF ARCHIVED RECORDING)
UNIDENTIFIED PEOPLE: Fillon, President, Fillon, President, Fillon, President.
SIEGEL: Fillon's conservative supporters, chanting Fillon, President, were jubilant. Bruno Jeanbart - not so much.
JEANBART: Yes, it was a bit disappointing.
SIEGEL: Since then, one French newspaper, La Parisienne, has decided to take a break from polling, not just because of bad French polls, but also because of U.S. polls in November and recent British polling. In fact, bad polling in the U.K. has hit a trifecta. First, in 2014, in Scotland's independence referendum, most British polls showed independence losing in a close race.
UNIDENTIFIED WOMAN: Should Scotland be an independent country? We're in favor of no.
SIEGEL: As viewers of Britain's ITV saw, independence did lose, but it sure wasn't close. The margin was nearly 11 points. Then, in 2015, Prime Minister David Cameron led his conservative party to a surprisingly decisive election victory that kept him in office. How surprising? Well, George Parker is political editor of The Financial Times.
GEORGE PARKER: The opinion polls, including the opinion polls that we were running in The Financial Times, were suggesting that David Cameron's chances of actually winning the election outright were very slim, indeed.
SIEGEL: Judging from the polls, if Cameron was to win, his party would still fall short of a majority in the House of Commons, and he would need a smaller party, the Liberal Democrats, to form a government. The morning after Election Day, that was not the news.
DAVID CAMERON: I've just been to see Her Majesty, the Queen, and I will now form a majority conservative government.
SIEGEL: Cameron's conservatives had one bid - an absolute majority of seats. The rest is history, and bad polling played a key role. In the days when the polls showed the election would be close, Prime Minister Cameron had promised the Eurosceptic wing of his own party that he would hold a referendum on leaving the European Union. It looked like a safe offer since the pro-Europe Liberal Democrats would surely demand a retraction of that promise as a condition for joining in a coalition. Now there would be no coalition.
PARKER: And as you say, the rest is history because David Cameron was then lumbered with the promise to hold a referendum on leaving the European Union, which he had to honor with disastrous consequences for him, at least, and, some would argue, for the country as a whole.
SIEGEL: Not to mention for what was left of the reputation of British polling. Most polls predicted Britain would stay in the EU. But when they announced the results in the city of Sunderland...
(SOUNDBITE OF PRESS CONFERENCE)
UNIDENTIFIED WOMAN: The total number of votes cast in favor of leave was 82,000.
(CHEERING)
SIEGEL: Those cheers, as streamed by The Guardian, were for leaving. The Brits voted 52 percent to 48 percent to quit the EU. How did 11 British polls do with that one? Sir Robert Worcester has been involved in British polling for decades.
ROBERT WORCESTER: Two of the 11 were internet polls who got it within one point. There was another one that had it 50-50, so they were two points out. So three of the eleven we can say got it right. But of the other eight, they were marginally on the wrong side.
SIEGEL: Worcester is a native Kansan who went to London and started a polling firm called MORI. He says the task of polling entails two objectives.
WORCESTER: You've got to get the people who are going to get out and vote and then how they're going to vote, so it's a two-stage effort.
SIEGEL: And as for that first step, The Financial Times' Parker says the British polls have done badly.
PARKER: The pollsters have been absolutely terrible at reaching the kind of people who are going to vote. And that's partly because, particularly in the case of Brexit, there were people voting who normally were seething with resentment, but couldn't be bothered to go out and cast their ballots. And so they were, in a way, unidentified by the pollsters.
SIEGEL: Bob Worcester sees a problem of evolving methodologies. He actually polled for the British government during the first referendum on staying in Europe back in 1975.
WORCESTER: We were doing face-to-face polls every day, but political polling is almost entirely now split between the phone and the internet. And it's taken quite a long time for the internet to climb in there and get themselves a model that can replicate the population statistically - the demographic profile of the United Kingdom.
SIEGEL: Back in France, how does pollster Bruno Jeanbart of Opinion Way explain his poll's 19-point error in the French primary? Well, Jeanbart says the novelty of a center-right primary open to left-wing voters made it especially tough to figure who would turn out. There was also a debate very late in the campaign that his poll missed. But he also adds this lesson - French voters don't think much of their political parties, and they don't feel very attached to them.
JEANBART: There's only 12 percent in France who have confidence in the political parties. It's very, very low. It means that people don't really believe so much in politics, and so they determine very late sometimes when they are going to turn out or not.
SIEGEL: Late deciders figured in November's polling hours in the U.S. presidential race, too. Courtney Kennedy runs surveys for the Pew Research Center, which did not publish an Election Eve poll this year. Courtney Kennedy says pollsters are examining several possible reasons for bad polling, not in national surveys, which were pretty accurate, but in certain battleground states.
COURTNEY KENNEDY: Particularly in Upper Midwest, there were some systematic, large polling errors in the same direction, which is quite concerning, for sure.
SIEGEL: In Wisconsin, Michigan and Pennsylvania, the polls had Clinton leading, and she lost. One factor Kennedy says pollsters are looking at is late deciders. In exit polls in those states, 12 percent of voters said they decided in the last week. Most of them went for Trump.
KENNEDY: And so if you conducted your poll, you know, in the middle or late of October, you would have missed that late movement.
SIEGEL: She says there are some other post-mortem questions for pollsters. Were Trump voters less likely to take part in polls or less likely to tell pollsters the truth? And when pollsters took their raw data and then weighted various demographic groups to create a model of the electorate, did they know what the electorate would look like? Courtney Kennedy of Pew says they didn't.
KENNEDY: A lot of the assumptions about what people who vote look like is based on the exit poll. And what we've learned is the exit poll data over-represent people that are college-educated, over-represent non-whites. And so, in fact, the people that came out to vote this year were somewhat more white and less educated than many pollsters had expected them to be.
SIEGEL: For every instance of bad polling, you can find unique circumstances that account for some of what happened. But there are also common themes in the U.S., France and Britain. Assumptions about party loyalties and likelihood of voting took a beating this year, and the failures of 2016 have put the very idea of predictive polling on notice for elections this year and next.
(SOUNDBITE OF MUSIC)
SHAPIRO: That's our colleague Robert Siegel. Transcript provided by NPR, Copyright NPR.
