Refining the science of public opinion polling

But then, so is almost everyone else, says the University of Wisconsin-Milwaukee (UWM) sociology professor, who has been president of the American Association for Public Opinion Research (AAPOR) since May.

The results of public opinion polls measure nearly every aspect of our lives today, from who we favor for public office to what kinds of cookies we prefer.

“You could give me a statistic and I’ll tell you the survey from which the data came,” says Mathiowetz. Surveys are used, for example, to determine statistics such as the U.S. unemployment and poverty rates, the percentage of people without health insurance and the Consumer Price Index.

Mathiowetz, who has experienced the full gamut of inquiries about polls from media – from Mother Jones to The Wall Street Journal – views them as necessary and useful tools. Although not perfect, she says, polls still are a reliable way for people to have their voices heard by lawmakers.

“They are what drive policymaking in a democratic society,” she says.

On the AAPOR’s executive committee, Mathiowetz keeps company with some high-profile names in the field of public opinion. Frank Newport, editor-in-chief of the Gallup Poll, who also blogs for USA Today; Mark Blumenthal, editor and publisher of Pollster.com.; and Scott Keeter, director of survey research for the Pew Research Center and an election-night analyst for NBC News, are a few.

“She’s been one of the most active and innovative presidents in a while,” says Richard Kulka, senior vice president at the social science research firm of Abt Associates Inc., and the AAPOR’s president-elect. “She’s been a major impetus for providing short courses and training, especially for journalists, who are the most immediate consumers of surveys. That’s her signature contribution.”

Accuracy

Well-designed surveys are generally reliable, Mathiowetz says. Even a relatively small sample size can yield accurate results if conducted properly. For example, a survey of 1,200 to 1,500 people is all that’s needed to gauge nationwide public opinion on a given topic.

But ultimately, individuals’ attitudes and behaviors are subject to change over even a short amount of time, making it difficult to capture a precise picture, she notes.

When the same poll can predict correctly for one major political party, but get it wrong for the other, there’s an issue that probably isn’t related to polling methodology overall, she says. In the 2008 New Hampshire Democratic primary, nine polls got the outcome wrong – predicting Hillary Clinton would lose big to Barack Obama. Yet the same polls accurately predicted John McCain would win.

Mathiowetz named a blue-ribbon committee to try to determine what factors might have affected the polls.

Though the committee’s findings are not yet complete, Mathiowetz says she suspects that one source of the problem was that pollsters overlooked a piece of their own research: that 25 percent of voters who voiced a preference for a particular candidate also said they could still change their minds.

“The role of undecideds in a close election is difficult to understand in advance,” she says. “After New Hampshire, the media have been careful to report the percentage of undecided voters, as well as the percentage of those who claimed a choice but also said they were not certain of their choice,” she says.

Credibility

Although polls have been common since early in the 20th century, the field is getting crowded. One of the goals of the AAPOR is to raise awareness of how polls are conducted and what makes them credible.

During this political season, Mathiowetz has taken on the challenge of warning the public about “push polls,” which she has condemned on the AAPOR Web site, on muckraker.com and in her blog on The Huffington Post.

“They are not polls,” says Mathiowetz. “They are political telemarketing in the guise of research. It may seem like a survey, but a push poll isn’t interested in your response. The goal of these calls is to influence voters through misinformation about candidates.”

Such tactics are designed to plant a seed of doubt in the minds of voters, she says. And strategically timed, they can reach thousands of people in a single night.

Power of suggestion

But even legitimate public opinion surveys are sensitive to the power of suggestion and are subject to multiple sources of error.

Mathiowetz’s own specialty, which she teaches at the graduate level, is improving the methodology of surveys to minimize those sources. (Her current research involves ways to make surveys of the elderly more accurate.)

It’s a field that encompasses psychology and other disciplines beyond statistics.

Issues that can influence survey-based statistics include the inclusion or exclusion of cell phones, the use of interactive voice recognitions systems (“digital interviews”) and participant non-response. Even a small detail, such as the order of response options – for example, the order in which you list presidential candidates – can have an impact on the way people respond.

When her term is up this May, Mathiowetz will take on a new role for AAPOR, serving as editor for the organization’s research journal, Public Opinion Quarterly.

Source: University of Wisconsin - Milwaukee