The Wiimote as an interface bridging mind and body

March 5, 2008

The Nintendo Wii is an immensely popular source of videogame entertainment, but more recently, it has been adapted for a number of different uses, such as a tool for physical therapy and as a form of exercise for geriatrics.

New research from the University of Memphis, published this week in the journal PLoS ONE, has found another use: psychological experimentation. By integrating the Nintendo Wiimote with a laboratory computer, psychologist Rick Dale and his student collaborators were able to extract rich information about a person’s reaching movements while they performed a learning task.

The authors were interested in how the dynamic characteristics of arm movement change as people become better at a task. Data from the Wiimote permitted the researchers to demonstrate that body movements change systematically along with change in mental processing (in this case, learning). These results provide new evidence that cognition and action systems, still thought by many to be relatively separate subsystems in the human mind, are actually deeply intertwined.

“The Wiimote is in fact the perfect interface to perform these kinds of experiments,” Dale remarked. “As the game itself is already designed to absorb a person’s body into the videogame experience, we just have to hook the Wiimote into a lab computer, and we can enjoy the rich streaming data that videogames typically use, but this time track them in experiments.”

Dale and his students continuously tracked the position and acceleration of participants’ choices as they learned to match unfamiliar symbols into pairs. As people learned, their bodies reflected the confidence of that learning. Participants moved the Wiimote more quickly, more steadily, and also pressed on it more firmly as they became familiar with the symbols. While everyone knows that you get better at moving in tasks that require intricate movement (such as learning to use chopsticks), these results suggest that your body movements are related to learning other information as well.

Their results suggest that when the body accompanies more complex learning experiences in school or at work, it can richly reflect that underlying process of learning. The authors suggest that this idea may help adaptive computer interfaces and learning technologies extract information about a user or learner – by paying close attention to their body dynamics.

The authors note that using the Wiimote now provides psychologists with a very affordable and immersive environment to study the relations between cognition and action. Existing technology to track three-dimensional movement typically costs many thousands of dollars, but the use of the Wiimote may provide an accessible and enjoyable alternative.

“One reason the Nintendo Wii is so wildly successful is that it integrates natural bodily movements with the mental processing involved in gaming,” Dale notes, “our results offer further testament to this. Your body and your mind are really one system, naturally changing with each other in all our daily learning and other cognitive experiences.”

Citation: Dale R, Roche J, Snyder K, McCall R (2008) Exploring Action Dynamics as an Index of Paired-Associate Learning. PLoS ONE 3(3): e1728. doi:10.1371/journal.pone.0001728 (http://www.plosone.org/doi/pone.0001728)

Source: Public Library of Science