Election polling methods constantly changing and improving, expert says

Editors' notes

This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

trusted source

proofread

With the presidential election just five weeks away, the only thing predictable about the campaign season is the daily churn of United States electorate polls.

Polls have garnered derision over the last few election cycles, but survey methodologists—including Kristen Olson, director of the University of Nebraska–Lincoln's Bureau of Sociological Research—are always working to improve the science.

Polling methodology is always changing, Olson, an expert in survey methodology, said. Most notably, pollsters have changed the ways polls are conducted over the last two decades. According to a study by the Pew Research Center, in 2000, polling was done almost exclusively by phone.

Today, pollsters rely on text, online and phone surveys, and probability panels that follow a group of voters through a set amount of time during an election campaign. More firms are also gathering the numbers, with the number of active polling firms more than doubled since the turn of the century.

Olson served on committees for the American Association for Public Opinion Research charged with reviewing polling methods and outcomes following presidential elections, which led to changes in how pollsters statistically weight their data and the transparency of firms' methods.

She served on the Public Opinion Research committee following the presidential elections of Barack Obama in 2008 and Donald Trump in 2016. The 2016 election stunned poll watchers, but Olson said when the committee dug into the data, the reasons for the outsized differences between the polls and the ultimate result became clearer.

"The real problems were in the state-level polls, where the estimates of anticipated vote choice were far off in important states," Olson said.

"One of the reasons for that was that state-level pollsters were not weighting for education, especially an overrepresentation of college graduates in their samples. One big recommendation in our report was to weight for education in addition to age, race and sex."

Olson said that the other issue was the way polling aggregators were modeling probabilities for election outcomes.

"There was a very large presence in the media and political discourse about these polling aggregators," she said. "We came down hard on them in our report because they were using inputs in their models with unknown errors, and they weren't really being transparent."

The American Association committees in 2016 and 2020 also examined how transparent pollsters were about their methodologies.

"The best polls are transparent about their methods," Olson said.

Poll watchers should keep in mind that any single poll is a snapshot in time, not necessarily predictive of an electoral outcome.

"It's really just showing what the electorate is thinking at a particular point in time," she said. "And there is a lot of evidence of late deciders—or those who are making their decision at the ballot box—and polls can't capture those in-the-moment decisions.

"The undecideds are pretty sizable, and there have been post-election surveys done that ask when a person made up their mind about who they were going to vote for, and large numbers of people reported very late decisions."

Additionally, despite efforts to incorporate different collection methods and statistical adjustments, polling is more difficult now than ever.

"Response rates are really low right now," Olson said.

"Response rates have been declining for 40 years—it's not a new thing—and they're generally declining in most countries. There's no singular explanation for it. There have been changes in norms of communication, changes in willingness to participate, changes in how we think about privacy, and more demands on our time."

So why should any attention be paid to polls?

"The world changes fast, especially in the world of politics," Olson said. "What people are thinking today might not be what they think tomorrow. When I look at a poll, I know that's what they were thinking yesterday, and having some sense of what people thought yesterday is informative, but it's certainly not the final answer."

And from a researcher's perspective, election polling can provide useful benchmarks for the field.

"As a survey methodologist, election polling is the most visible thing we do, even more so than the census, more than the unemployment rate," Olson said. "And they are one of the few places where a true value is known, and we can evaluate how well our methods worked, and if what we're doing is getting us closer to the truth.

"Pollsters are often at the forefront of innovation in methods, and it's important to keep evaluating and studying those methods."

Provided by University of Nebraska-Lincoln