A new direction for game controllers: Prototypes tug at thumb tips to enhance video gaming

University of Utah engineers designed a new kind of video game controller that not only vibrates like existing devices, but pulls and stretches the thumb tips in different directions to simulate the tug of a fishing line, the recoil of a gun or the feeling of ocean waves.

"I'm hoping we can get this into production when the next game consoles come out in a couple of years," says William Provancher, an associate professor of mechanical engineering who is in Vancouver, British Columbia, demonstrating the new game controller with his students March 5-7.

They are demonstrating the device and presenting studies about it during the Institute of Electrical and Electronics Engineers' Haptics Symposium. Haptics deals with research about touch, just as optics deals with vision. A patent is pending on the device.

The first haptic or touch feedback in game controllers came in 1997 with the Nintendo64 system's "rumble pack" that makes the hands vibrate using an off-balance motor to simulate the feel of driving a race car on a gravel road, flying a jet or dueling with Star Wars light sabers, Provancher and colleagues write.

His new controller does something additional: it delivers directional cues to the player by stretching the skin of the thumb tips in different directions.

"We have developed feedback modes that enhance immersiveness and realism for gaming scenarios such as collision, recoil from a gun, the feeling of being pushed by ocean waves or crawling prone in a first-person shooter game," Provancher says.

The latest game controller prototype looks like controllers for Microsoft's Xbox or Sony's PlayStation but with an addition to the controller's normal thumb joysticks, on which the thumbs are placed and moved in different directions to control the game. In the new controller, the middle of each ring-shaped thumb stick has a round, red "tactor" that looks like the eraser-head-shaped IBM TrackPoint or pointing stick now found on a number of laptop computer brands.

If a gamer's avatar runs into a wall, the tactor under the thumb moves back to mimic impact. Both tactors can move from side to side to mimic ocean waves. And when a fish bites in one of the games the researchers tried, "as the fish jerks on the line, you can feel the tactor jerk under your thumb," Provancher says.

Video games commonly are designed so the left thumb stick controls motion and the right controls the player's gaze or aim. With the new controller, as a soldier avatar crawls forward, the player pushes the left thumb stick forward and feels the tactors tugging alternately back and forth under both thumbs, mimicking the soldier crawling first with one arm, then the other.

Provancher also hopes to adapt the new game controller design for use as a smart phone peripheral device. A phone would fit into the device with game-controlling thumb sticks and tactors on each side of the phone.

Of Thumbs and the New Game Controller

Smart phones provide an analogy for the new advance in game controllers, Provancher says. Early smart phones used vibrations only when the phone rang. Current models use more sophisticated feedback: a vibration with each touch to help users improve their accuracy when they use a touch-screen keyboard.

Provancher's previous research showed that skin-stretch devices on a steering wheel could stretch the index finger tips left or right to tell drivers to turn left or right – and did so just as accurately as a navigation system's computerized voice. His latest research is more than fun and games because "by placing skin-stretch feedback in a game controller, it creates a nice testing environment for understanding human perception and cognition," he says.

Indeed, a critical question in determining if the new game controller would work dealt with how players perceived front-back and left-right tugs at their thumb tips when the thumbs were angled inward as they are in a typical Xbox or PlayStation controller.

A study analyzing how gamers perform with the new, thumb-tip feedback device was conducted by Provancher, computer science doctoral student Ashley Guinan, computer science master's degree student Rebecca Koslover and mechanical engineering master's student Nathaniel Caswell. The research was funded by the National Science Foundation and the University of Utah.

The study, which Guinan was presenting during the Haptics Symposium, found that gamers' brains made the necessary mental rotation of the directional tugs so they could perform just as well if their thumbs were angled as if their thumbs were straight.

Provancher says the angled thumb position is better ergonomically, both in terms of comfort and of ease in pushing forward with angled thumbs.

He says that his future research will focus on details of how to provide skin-stretch feedback effectively at the same time a game also produces sounds, sights and vibrations.

Provancher's earlier study of the steering wheel device that told the fingertips which way to turn showed that information can be conveyed by touch even when auditory and visual information also are present.