For computer scientists exploring face recognition, the question is 'who?'

February 7th, 2007

The images on the right are original images of Richard Nixon and Winona Ryder. Most people have a better chance of recognizing a person with eyebrows and no eyes (center), rather than with eyes and no eyebrows (left). Credit: Sinha, et al.

One of the most challenging tasks for computer vision researchers is to design a system that can automatically recognize individual faces. Humans, who can perform this task in as little as 50 milliseconds—almost unconsciously, it seems—may even be pre-programmed from birth. But while computers currently lag behind their biological counterparts, scientists hope that one day, machines will not only equal the human ability of facial recognition, but exceed and, in turn, improve human visual processes.

“The human brain has an amazing ability to recognize faces,” scientist Richard Russell of Harvard told PhysOrg.com. “It would be a waste not to learn from it, especially since there are no other computer strategies so far that come close to the kind of face recognition performance the human brain exhibits.”

Along with Russell, MIT researchers Pawan Sinha, Benjamin Balas, and Yuri Ostrovsky all have a strong interest in computer science but spend most of their time studying the human brain. In their recent study, “Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About” in the Proceedings of the IEEE, the group compiles 19 vignettes of how humans recognize faces.

Enlarge

Research shows that humans can recognize familiar faces even with high degradations. From left to right: Michael Jordan, Woody Allen, Goldie Hawn, Bill Clinton, Tom Hanks, Saddam Hussein, Elvis Presley, Jay Leno, Dustin Hoffman, Prince Charles, Cher, and Richard Nixon. The average recognition rate at this resolution is one-half. Credit: Pawan, Sinha, et al.

Although they are careful to point out that vision systems would not “slavishly imitate” humans’ perceptions, the scientists also include pointers on how each of the 19 results could influence the design of an automatic face-recognition system.

One interesting overall impression from the research is that humans do a remarkable job of identifying faces, even under many kinds of extreme degradations. For example, humans can recognize familiar faces in very low-resolution images (Result 1). The scientists speculate that we have learned to do this because we must often recognize people at a distance and in motion. Unfamiliar faces, however, were more difficult with degradations—which may explain why it’s difficult to identify unknown suspects in a police line-up (Result 2).

Further, we don’t need to see every feature of a face to identify a person, and we can quite easily recognize faces with heavy distortion along the length or width axis (or in funhouse mirrors) (Result 6).

“Human recognition not only performs well under severely degraded conditions, but even under the best circumstances our visual system doesn't seem to bother storing perfect models of the objects we see,” explained Balas. “Recognition may be less a problem of finding ways to recover the world perfectly from the image, and more a problem of finding quick and dirty solutions that work most of the time.”

For humans, the scientists explain that these abilities may be due to the nature of how the brain processes human faces: facial features, unlike many other objects in our world, are processed holistically rather than piecemeal (Result 4). However, the brains of young children may utilize a piecemeal strategy, which progresses to holistic by the age of about 10 years (Result 16).

The single facial feature that is most vital for recognition is a bit of a surprise—it’s not the eyes, but rather the eyebrows that people seem to depend upon most for identification (Result 5). And although it makes sense that facial shape plays an important role in recognition, research shows that skin pigmentation—even slight shade differences—plays at least as important a role as shape (Result 9). These areas, Sinha et al. suggest, might receive valuable attention from computer designers.

“The primary emphasis in the last few years has been security: finding terrorists and criminals, securing access to sensitive physical locations,” said Ostrovsky. “But there are many other fun or practical applications as well: finding your friend in a crowded stadium or automated labeling of celebrities or politicians in news broadcasts. Another big future application is instant personalization of settings for your electronic devices (e.g., computer, mobile phone, ‘smart home’). We do a lot of things differently depending on who is around us, and electronic devices should do the same without our having to explicitly login every time.”

Because the adult brain processes faces holistically, it is easier for most people to recognize these two faces in the right image, where they are misaligned. They are Woody Allen and Oprah Winfrey. Credit: Sinha, et al.

Because humans generally do not have a problem recognizing faces under varying illumination, with chaotic background noise, or at long distances, computer systems may benefit from learning the mechanisms of the brain. As research has suggested, our brains have likely devoted a specialized neural resource solely for facial recognition—perhaps computers could use a similar “brain region” to process faces. But then again, maybe computers will find an even better way.

“Too much generalization can prove to be a short-coming in situations where individuals are actively trying to deceive and adopt other people’s identities,” explained Sinha. “In these circumstances, a detail-oriented scheme, say examining the precise pattern of irises or the exact distances between facial features, might be more appropriate, despite being implausible as human strategies.”

Citation: Sinha, Pawan, Balas, Benjamin, Ostrovsky, Yuri, and Russell, Richard. “Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About.” Proceedings of the IEEE. Vol. 94, No. 11, November 2006.

By Lisa Zyga, Copyright 2007 PhysOrg.com.
All rights reserved. This material may not be published, broadcast, rewritten or redistributed in whole or part without the express written permission of PhysOrg.com.