Too many choices can spoil the research

Authors Jordan J. Louviere (University of Technology, Sydney), Towhidul Islam (University of Guelph, Ontario), Nada Wasi, Deborah Street, and Leonie Burgess (all University of Technology, Sydney) examined choice experiments, where researchers study which brands or products consumers prefer. The research found that experiments that are considered "statistically efficient" (asking complex questions of fewer respondents) lead to less consistency in participants' choices.

"The likely price a researcher pays in using optimal designs is less consistent choices," write the authors.

The authors constructed experiments where participants had to choose among a number of options for ordering pizza or choosing vacations. They designed 22 different questionnaires with varied amounts of attributes. They found that the more efficient the study design was, the less consistent participants were with their choices.

The most efficient designs use many different attributes (such as delivery time and quality of ingredients for pizza). The authors believe that highly efficient study designs impose a higher cognitive load on the participants (requiring more thought for each response). That's why their responses decrease in consistency.

"Our results suggest that researchers should pay attention to the ways that they design or administer experiments because these decisions can impact choice outcomes and choice variability," write the authors.

Source: University of Chicago