Presidential primary 2008 polls: What went wrong

March 30, 2009

University of Michigan survey experts working with the American Association for Public Opinion Research have identified several reasons polls picked the wrong winners in the 2008 Presidential Primary.

The study is believed to be the most comprehensive analysis ever conducted of presidential primary polls.

"The most jarring element of the presidential primary polling was that polls picked the wrong winner in New Hampshire," said U-M polling expert Michael Traugott, who chaired the AAPOR committee composed of leading academic and private sector experts in public opinion and survey research. "We wanted to find out why."

The results of the committee's analysis show that a handful of methodological missteps and miscalculations combined to undermine the accuracy of predictions about presidential primary winners in New Hampshire and three other states.

One source of error the researchers were able to eliminate was the so-called 'Bradley Effect,' in which people say they support a Black candidate in order to appear unbiased, but then cast their ballots for a white candidate in the privacy of the voting booth.

"Many New Hampshire polls predicted would beat Hillary Clinton in that state," said Traugott. "So when Clinton won, some people pointed to latent racism as the reason. But in the data we have from a wide variety of New Hampshire pre-election and exit polls, we found no evidence that whites over-represented their support for Obama."

For the report, supported in part by a grant from the U-M Institute for Social Research (ISR), Traugott and colleagues analyzed individual, household-level response data provided by seven polling organizations. They also compared information on question wording, weighting, interviewer characteristics, sampling frames, and other methodological issues from up to 19 other firms, in many cases relying on publicly available information gleaned from the Internet.

"This analysis suggests some important explanations for the errors in the 2008 New Hampshire Presidential Primary and raises significant questions about pre-election polling methods," said Richard Kulka, AAPOR president.

"The materials we received from polling organizations showed that there was much more variation in the methodology of pre-election polls than I ever imagined there would be," Traugott said.

The committee analyzed performance in four primary states: Wisconsin, South Carolina, California, and New Hampshire. Although the limited data they received made it impossible to conduct definitive tests of all likely sources of different poll performance, the following factors were identified as the most likely reasons the polls got it wrong:

• The New Hampshire primaries occurred only five days after the Iowa caucuses, truncating the polling period in New Hampshire.

• Most commercial polling firms conducted interviews on the first or second call, but respondents who required more effort to contact were more likely to support Clinton. Instead of reworking their initial samples to reach these hard-to-contact people, pollsters typically added new households to the sample, skewing the results toward the opinions of those who were easy to reach on the phone, and who typically supported Obama.

• Non-response patterns, identified by comparing characteristics of the pre-election samples with the exit poll samples, suggest that some groups that supported Clinton---such as union members and those with less education---were under-represented in pre-election polls, possibly because they were more difficult to reach.

• The order of candidate names on state primary ballots likely contributed to increased support for Clinton in New Hampshire, where her name appeared near the top of a long list of names and Obama's appeared near the bottom.

Traugott noted that the analysis also revealed wide variation in the primary question respondents are asked---the so-called trial heat question about which candidate they prefer in the coming election. In New Hampshire, there were 11 different question wordings used in the Democratic primary, and 10 different wordings used in the Republic primary. In some versions, the candidates' names were not mentioned at all. In others, only the "major" candidates were named. Some polls randomized the candidates' names.

"We also learned that some polling firms are buying lists of registered voters with phone numbers, and then they are contacting people with interactive voice response technology---basically computerized calls---and that they're taking information from the person who answers the phone which may or may not be the person identified in the sample," Traugott said. "This should be a focus of further research." Another firm interviewed any registered voter in the household.

Source: University of Michigan

Filter


Move the slider to adjust rank threshold, so that you can hide some of the comments.


Display comments: newest first

Gunzo
Mar 30, 2009

Rank: not rated yet
Boil results down to their 'add value' level. Not enough people were asked in the polls leading up to the vote.
Rank 2 /5 (1 vote)
Relevant PhysicsForums posts
  • Can I forget a language?
    createdFeb 10, 2012
  • The Biggest Lie Ever
    createdFeb 09, 2012
  • What are the limits of learning?
    createdFeb 06, 2012
  • Isn't that grammatically wrong?
    createdFeb 06, 2012
  • What does it mean when traders are indifferent?
    createdFeb 04, 2012
  • Peak of Our Civilization
    createdFeb 04, 2012
  • More from Physics Forums - Social Sciences

More news stories

A frank discussion of the power law and linking correlation to causation

(PhysOrg.com) -- Michael Stumpf a mathematics professor at Imperial College in London, and Mason Porter a lecturer at Oxford have teamed together to write and publish a perspective piece in Science regarding the in ...

Other Sciences / Mathematics

created Feb 10, 2012 | popularity 5 / 5 (5) | comments 10 | with audio podcast report

Employers feel no love for unscrupulous practice of 'service sweethearting'

A new study led by two Florida State University marketing professors finds that some frontline service employees who are rewarded for hikes in customer loyalty and satisfaction also may engage in "service ...

Other Sciences / Economics & Business

created Feb 10, 2012 | popularity 3.3 / 5 (3) | comments 10

US workers are 'giving away the store,' costing firms billions

Nearly 70 percent of the nation's service employees give away free goods and services – from hamburgers to cable TV – costing companies billions of dollars a year, according to a groundbreaking study.

Other Sciences / Economics & Business

created Feb 09, 2012 | popularity 3.5 / 5 (4) | comments 10

New insights into how to correct false knowledge

The abundance of false information available on the Internet, in movies and on TV has created a big challenge for educators.

Other Sciences / Social Sciences

created Feb 07, 2012 | popularity 4.9 / 5 (7) | comments 9 | with audio podcast

Neanderthal demise due to many influences, including cultural changes: study

As an ice age crept upon them thousands of years ago, Neanderthals and modern human ancestors expanded their territory ranges across Asia and Europe to adapt to the changing environment.

Other Sciences / Archaeology & Fossils

created Feb 07, 2012 | popularity 4.4 / 5 (5) | comments 8 | with audio podcast


Scientists discover molecular secrets of 2,000-year-old Chinese herbal remedy

For roughly two thousand years, Chinese herbalists have treated Malaria using a root extract, commonly known as Chang Shan, from a type of hydrangea that grows in Tibet and Nepal. More recent studies suggest that halofuginone, ...

New method to examine batteries -- MRI from the inside

There is an ever-increasing need for advanced batteries for portable electronics, such as phones, cameras, and music players, but also to power electric vehicles and to facilitate the distribution and storage of energy derived ...

Google might launch Drive for cloud storage soon

(PhysOrg.com) -- Google's next big move, according to the Wall Street Journal, is a cloud storage service called Drive. Hardly first to the plate, Google is simply catching up to introducing its cloud reposi ...

Lab study raises questions over nano-particle impact

Tests involving chickens have raised questions about the impact on health from engineered nano-particles, the ultra-fine grains commonly used in drugs and processed foods, scientists said on Sunday.

A mitosis mystery solved: How chromosomes align perfectly in a dividing cell

Although the process of mitotic cell division has been studied intensely for more than 50 years, Whitehead Institute researchers have only now solved the mystery of how cells correctly align their chromosomes during symmetric ...

Starve a virus, feed a cure? Findings show how some cells protect themselves against HIV

A protein that protects some of our immune cells from the most common and virulent form of HIV works by starving the virus of the molecular building blocks that it needs to replicate, according to research published online ...