New sequential decision making model could be key to artificial intelligence

November 8, 2006

“Decision making,” Mikhail Rabinovich tells PhysOrg.com, “is everywhere, and not just with humans. Animals use it, and robots do. But the traditional approach to decision making is too simple.”

Rabinovich and his colleague at the Institute for Nonlinear Science at the University of California, San Diego, Ramon Huerta, along with Valentin Afraimovich at the Institute for the Investigation of Optical Communication at the University of San Luis Potosi in Mexico, present a new model for understanding decision making. Their paper, titled “Dynamics of Sequential Decision Making,” has been published on Physical Review Letters.

The idea behind sequential decision making is fairly simple: if an intelligence has to decide between two items, something will follow, based on the decision made. “In the traditional, simplistic model,” says Rabinovich, the decision maker has to answer a simple question — left or right for instance — choosing between two attractors.” This results in a simple “if-then” equation. However, when real decision making is in question, there is more than a simple “if-then” at work. “In reality,” he says, “it’s much more complex and interesting.”

Rabinovich explains that a sequential approach is needed: an approach that combines dynamic and probabilistic steps. And that, he says, is precisely what he, Huerta and Afraimovich are proposing. “This is a new class of model,” he says. “We have found a window to consciousness, and now we can generalize this into other cognitive functions.” He lists sequential attention, working memory and planning as other cognitive areas that could be benefited by this research into sequential decision making.

But it’s not just the knowledge of how we humans might do something that makes Rabinovich and his colleagues’ work so interesting. “This can be applied to the artificial brain,” Rabinovich insists. “If we are going to create an intelligent brain for a robot, we have to think of these independent elements.” Basically, a process is needed for modeling a robot brain that could work as a human or animal brain. This is a model for getting there: “We have to be able to answer these questions in a qualitative way.”

“If you are talking about an animal trying to flee from a predator, you have to look at the complex landscape. The prey has more than one decision to make. It has to decide which way to go not only at this moment, but at each moment for the length of its life. And there are other factors that influence the decision as well.” Rabinovich continues his illustration: “The same is with a robot that we put on another planet. It has to make a decision at many critical moments, not only about direction, but also about speed, whether or not to go, and other decisions at each decision making moment.”

The paper describes this new class of models using differential equations with finite possibilities, as well as suggesting rules for solving uncertainty. Rabinovich, Huerta and Afraimovich use an approach based on competition between cognitive states that are possible with the use of transient dynamics. Environmental decision making criterion are used to model how the sequential steps of the process are controlled. The paper finds that the length of life for the process depends on the number of choices, as well as the cognitive agents involved which both characterize the level of system intelligence.

Rabinovich clearly sees many benefits in further developing this work. “This is important,” he says. “Any kind of real life needs some kind of sequential decision making activity.”

By Miranda Marquit, Copyright 2006 PhysOrg.com