Virtual Eve: first in human computer interaction

Eve is what is known in the information sciences as an intelligent or affective tutoring system that can adapt its response to the emotional state of people by interaction through a computer system.

The system “Easy with Eve” is thought to be the first of its type.

The ability of virtual Eve to alter her presentation according to the reaction of the child facing her at the keyboard has been hailed as an exciting development in the $25 billion e-learning market.

The Massey scientists, led by Dr Hossein Sarrafzadeh at the Auckland-based Institute of Information and Mathematical Sciences, tell the story of creating Eve and the teaching system in the latest issue of the leading international journal on information sciences, Elsevier.

Because one-to-one teaching is known to be the most effective teaching method, Dr Sarrafzadeh says the researchers wanted to create a virtual teacher that could pick up body language and facial expressions – like a real teacher – to interact and to ensure they are holding the attention of students.

He says the realisation that software systems would significantly improve performance if they could adapt to the emotions of the user has spawned research and development in the field of affective or intelligent tutoring systems.

“With rising demand for long-distance learning and online tutoring, a computer programe capable of detecting human emotions may become a critical teaching tool.”

Although Eve was developed for one-to-one maths teaching with eight-year-olds, she is a significant new character in the future of human computer interaction and could be a personalised virtual tutor by any name.

Linked to a child via computer, the animated character or virtual tutor can tell if the child is frustrated, angry or confused by the on-screen teaching session and can adapt the tutoring session appropriately.

The animated Eve (with a human-sounding voice) can ask questions, give feedback, discuss questions and solutions and show emotion. To develop the software for this system the Massey team observed children and their interactions with teachers and captured them on thousands of images.

From these images of facial expression, gestures and body movements they developed programs that would capture and recognise facial expression, body movement, and (via a mouse) heart rate and skin resistance.

The system uses a network of computer systems, mainly embedded devices, to detect student emotion and other significant bio-signals.

“When we interact with people we expect them to take note of our feelings and reactions. Soon we will be able to expect the same from a computer,” says Dr Sarrafzadeh.

The introductory video of virtual Eve is available online: http://news.massey … ve-intro.mov

Source: Massey University