On track with the latest iTV moves

January 21, 2005

Demonstrating how truly interactive television can be, GMF4iTV’s system allows producers link rich media content to moving objects in normal TV programmes, changing telespectators from passive to active viewers.
"The [GMF4iTV] project is dealing with the complete broadcast chain, from the content side to the end-user side," explains the coordinator Gerhard Stoll of Germany’s Institut für Rundfunktechnik, a project partner.

“The entire system is based on existing standards,” he says. These include the Multimedia Home Platform (MHP) and MPEG standards. “There is nothing proprietary.”

Taking a plain MPEG-2 video, the content provider decides to associate some additional information to a predetermined object for interaction. Such information can comprise MPEG-4 audiovisual clips, MHP applications, images or HTML. Once selected, the object can be tracked in a semi-automatic way throughout the programme. “Tracking can work either forward or back,” adds Stoll, “so the provider can even start in the middle of a scene.”

From the tracking and annotation an MPEG-7 metadata description is generated that contains information about the tracked objects, their movements and the association with additional content.

According to Stoll, for a broadcaster to produce such interactive content, the speed of the tracking depends on the speed of the computer, so it can take 20-30 times the real-time to play. It also depends on the size and number of objects, and the time taken to associate the pre-prepared additional content. But, as he points out, “familiarity with the system can also improve production speeds.”

Together with the associated Rich Media content, the dynamic MPEG-7 metadata is then transmitted synchronously to the video sequence, and received by a MHP Set-Top-Box (STB).

The STB provides the viewer with the functionality of a new and easy to use graphical user interface to enjoy the interactive content of the object that appears surrounded by a coloured rectangle. Viewers can either access the highlighted content by pressing the appropriate button on their TV remote control or, thanks to a wireless connection between the PDA and STB, by touching the highlighted object on the PDA screen. The viewer can decide, on which device to view the additional content. Either on TV, on the PDA or on both. Even watching different additional content on multiple PDAs is possible. This multi-viewer functionality solves today’s problem of interactive TV, the lack of support for interactivity for more than one viewer.

The IST-project GMF4iTV’s system also supports personalisation. The producer can decide what objects appear in which shots and what type of additional content to display. If this matches the user profile stored on the STB, the interactivity appears. This personalises the interaction. For example, watching a nature documentary, a child might be presented with a simple quiz while an adult watching the same programme would receive additional detailed information about the animal’s behaviour.

Alternatively, watching a fashion show, male models could be highlighted for male viewers while women could select female models. The viewer could also be asked at the start of the programme whether he/she prefers shopping options or designer information, and the choice would be stored in the user’s STB profile.

Although the three demonstrations, produced by GMF4iTV, relate to sports, music and fashion it could be applied to any content. From documentaries to educational programmes to shopping opportunities, for example. “It depends on the interests of the content provider who has to think how to use this new way of direct interactivity with certain objects and what can be added, and how to derive revenue from viewers,” Stoll says.

Presented at Amsterdam’s IBC 2004 and Munich’s Medientage 2004 conferences, the GMF4iTV’s demonstrations generated a lot of interest among attendees, says Stoll.

“You cannot use it [the system] in real-time because of the authoring process,” notes Stoll. “It is not possible at the moment. Perhaps later, with faster computers and better object tracking,” he adds.

Source: IST Results