Searching for an Unfriendly Face

September 26, 2006

Photo Courtesy United States Army

As a soldier scans the crowded streets of Baghdad, so do another set of eyes. The second set, located on the soldier's rifle, belongs to a camera system that instantly recognizes the faces of potential threats.

The camera's brain, a two-pound computer system worn on a belt or in a pack, is the first portable video system capable of matching faces with known threats in real time. It's being developed by Frederick-based ACAGI Inc., along with researchers at the University of Maryland's A. James Clark School of Engineering, through Maryland Industrial Partnerships Program funding.

"Soldiers have cameras on their rifles or helmets, but that's only half the equation. They don't recognize anything," said ACAGI CEO Peter Spatharis. "If our system sees somebody it knows, it tells you, and it tells others so quick decisions can be made."

ACAGI's Image Acquisition and Exploitation Camera System (IAECS) is loaded with previously taken photos of potential threats. As a soldier's camera pans over faces, the system converts them into digital images and compares them with the stored photos. Rama Chellappa, a professor in the Clark School's Department of Electrical and Computer Engineering, developed one of the face recognition technologies ACAGI will use. Electrical and computer engineering graduate student Raghuram Gopalan will put Chellappa's technology into the company's system. ACAGI will license this technology from the university.

Chellappa's face recognition occurs in three steps. First, the system searches for oval shapes and detects that a face exists in a video stream, and saves it as a digital image made up of tiny dots, or pixels. The system then creates a simplified map of the enclosed face based upon the light values of each pixel. Lastly, it compares the newly created pixel map with existing photos, and selects a similar match, if one exists.

"Identifying a face in a crowd is harder than trying to find a needle in haystack," said Chellappa. "We're trying to find a piece of hay in a haystack. Our job is to make that piece of hay as easy to find as a needle."

While the IAECS will store tens of thousands of known face records, it also categorizes new faces it sees and saves them for later, an operation the Department of Defense calls--marking and tagging--its second-highest priority next to finding improvised explosive devices,according to Spatharis.

"A person who appears to be a soldier's friend may be his enemy tomorrow," said Spatharis. "We're giving users the ability to recognize somebody and say 'where have I seen him before?' If soldiers encounter trouble at two different locations where the same face is seen, it's obvious there is a connection."

Ankur Srivastava, an assistant professor in electrical and computer engineering and an expert in embedded computing systems, will optimize the IAECS's software and hardware to perform on-the-fly, increasing the system's power performance by as much as 40 percent. ACAGI plans to release the first IAECS prototype by the end of December.

Additional product features will include three-dimensional sound, enabling soldiers to tell where a sound, or a round of fire, came from, as well as Global Positioning System location, giving users the ability to collect local image data with the camera and match it with GPS information to figure out where they are.

Spatharis also plans to target broadcast media with his product. The IAECS could help video broadcasters index and file footage as they take it, marking when people speak (through face recognition), and using speech-to-text technology to catalog whats being said and when.

Source: University of Maryland, By Eric Schurr