Virtual Voices

April 15th, 2005

Wearable Captioning System to Make Public Venues Accessible to People who are Deaf or Hard of Hearing

For people who are deaf or hard of hearing, the voices of actors, teachers, sports announcers and clergy are often silenced.
With captioning typically available only in selected movie theaters, a lack of access to information often limits community involvement for people who are deaf or hard of hearing. But a wearable captioning system developed at the Georgia Tech Research Institute (GTRI) and recently licensed to a metro Atlanta company may change that situation for the 28 million Americans (about 10 percent of the population) who are deaf or hard of hearing.

Image: A wearable captioning system developed at the Georgia Tech Research Institute (GTRI) relies on mobile wireless technology to allow people who are deaf or hard of hearing to receive information that is being presented audibly to the general public in a variety of public venues. Georgia Tech Photo: Gary Meek

The system, which relies on mobile wireless technology, will allow users to easily receive information that is being presented audibly to the general public in a variety of public venues, such as movie theaters, museums, playhouses, schools, government meetings, sports arenas, transit stations and places of worship. Because the system can transmit multiple text streams, it may also be used for language translation; its software is internationally compatible. Also, the system could transmit optional information that a hearing patron might want, such as statistics at a sporting event.

"The venue must generate the captions for patrons, but our system is an easy way to get captioning to patrons in a way they want - customized for each person," said GTRI project director Leanne West. "Right now, captioning is typically available only in movie theaters - and just a limited number of them - but this device gives us a way to deliver captioning at any venue."

Peacock Communications Inc., of Marietta, Ga., plans to sell the captioning system software called COMMplements™ and consult with venues on hardware and installation issues, said CEO Jack Peacock. He is seeking investor funding to finalize enhancements of the software. Peacock will market the software in the United States initially and then in Canada and western Europe.

In COMMplements, captions are sent by a venue's transmitter via standard wireless technology - the 802.11b wireless protocol - to a receiver device, such as a personal digital assistant (PDA), which also displays captions, West explained. The 802.11b wireless technology is already installed in many places, such as baseball stadiums, coffee shops and restaurants, and even urban business districts.

The venue operates the transmitter, and the patron can borrow the receiver and display from the venue or bring their own, West said. Wireless-enabled PDAs and laptop computers can serve as receivers and displays, or patrons can use a micro display that plugs into a PDA and attaches to their glasses or is worn on a headband. Although positioned close to the eye, the micro display uses optics that make its screen appear to float several feet away, giving users relaxed viewing of text seemingly overlaid on their visual field.

Captions can be pre-recorded or text can be generated in real time with a shorthand typing method such as CART (Communication Access Real-Time Translation). As voice recognition technology improves, captioning could be derived in this way, West added.

Before West and her colleagues began the project, she consulted with the Georgia Council for the Hearing Impaired to gauge the need for and interest in a new type of captioning system. With their interest confirmed, the project got funding from GTRI and a grant from the Wireless Rehabilitation Engineering Research Center (RERC) at Georgia Tech. The RERC is funded by the National Institute on Disability and Rehabilitation Research, a unit of the U.S. Department of Education.

With an early prototype in hand, researchers tested the system on two different occasions, including the annual Self-Help for Hard of Hearing People (SHHH) conference. The tests included 63 volunteers who were deaf or hard of hearing and ranged in age from 15 to 75.

"We wanted to make sure we were on the right track and including features they wanted and felt were necessary," West said. "It doesn't help to build something if the person you're building it for doesn't want to use it. That is often overlooked when people are developing assistive technologies."

In their tests, researchers simulated a movie theater setting, playing anywhere from 15 to 90 minutes of video. Participants wore glasses with a micro display attached.

Most volunteers reported that the display "took some getting used to," but 65 percent of users said it took less than 10 minutes to get the display in comfortable reading position. Also, 65 percent said the text was easy to read.

"We took their comments and included some of the features they wanted to see," West said. "Some people wanted an option for the font style, but we decided for several reasons to use the Tiresias font developed by the Royal National Institute for the Blind for people with low vision. Now, we get lots of compliments on the font."

West and her colleagues also created customizable features such as text size and color, background color and focus adjustment.

Most study participants -- 84 percent -- said they would not feel self-conscious about wearing the communication assistant in a public venue. Subjects called the system "neat," "truly amazing" and "worthwhile." One person commented: "This is a good device to carry around in case an interpreter doesn't show up."

The idea for the captioning system developed about five years ago out of a luncheon discussion session in West's research division at GTRI. Participants were discussing the litigation - now settled - over the lack of captioning in some movie theaters. The group agreed that there must be a better way to provide captioning and to offer it in a variety of venues.

Right now, the leading technology for providing captions in a movie theater is called the Rear Window® Captioning System, which was developed by the National Center for Accessible Media. The system is available in some movie theaters and displays reversed captions on a light-emitting diode (LED) text display mounted in the rear of a theater. Patrons who are deaf or hard of hearing use transparent acrylic panels mounted in their seat cup holders to reflect the captions so that they appear superimposed on the movie screen.

West said the GTRI system is more accommodating than the Rear Window system. The GTRI system handles multiple text streams, is fully customizable and discreet, and can be used in any public venue willing to provide captioning. Also, it works on a personal electronic device that users can affordably own and easily carry with them. The Rear Window system requires patrons to borrow equipment from the movie theater.

GTRI researchers are now enhancing the software with security measures to prevent hacking and are updating it to accommodate a newer, more rugged, and less expensive micro display. They also plan to create a customized version of the captioning system for venues to distribute to patrons who don't have a PDA.

Source: Georgia Institute of Technology