Semi-automatic multimedia monitor tracks sponsorship

Developed within the IST project DIRECT INFO, a prototype of the Web Services-based system was successfully tested in the sports arena. “We tracked sponsorship of the Italian football team Juventus,” says Herwig Rehatschek, the project coordinator, “looking at videos of the games played and relevant press articles.”

Sponsorship tracking involves monitoring lots of media, which is normally done manually or not at all. Machines struggle to do this because they must get the context (e.g. sport events in connection with the sponsored team) right and exclude analysis of traditional commercials from TV programmes. Commercials are a different marketing instrument and have to be regarded separately from sponsorship analysis.

Promising results
The project results are promising. Up to 90 per cent of the logos visible in videos were automatically detected, though this dropped to 24 per cent for logos on players’ shirts. Being printed on non-rigid and moving objects, shirt logos are much harder to detect and track – unlike logos on rigid walls or billboards.

“Our system produces a list of appearances relevant for the customer, including logos (time, size and position on the screen), positive and negative mentions of the sponsor and important topics raised,” adds Rehatschek. Media analysts sell this information to their clients. The project system is aimed at media-information firms in advertising, news, music and sport-sponsorship monitoring in particular. They want to know how often a specific brand is ‘mentioned’ on TV, in the electronic media and press to use it to generate reports for decision-makers.

Semi-automatic monitoring
Sponsorship is monitored with a fixed workflow. This first involves the ‘acquisition’ of material – including video of TV programmes and PDF versions of newspaper articles. To determine if the material is relevant for sponsorship tracking, it is automatically filtered by genre (music, sport, etc.) and by looking at the electronic programming guide for TV programmes. The system then identifies ‘semantic blocks’ (the relevant programmes) for further analysis, including detection of logos, topic detection and multi modal event analysis. It can even decide from a speech transcript or a PDF text if the sponsor is mentioned in a positive, negative or neutral way.

Analysis results can be checked manually for quality – and edited or redone if necessary. The last stage involves fusion of the semantic data into a database and the generation of relevant appearances. For each customer, a media analyst can prepare specifically tailored reports via a special set up application. Customers can query these reports via a standard Web browser, to look at relevant appearances of a logo and various statistics.

“Our project’s main innovation was identifying semantic information in audiovisual media,” says the coordinator.

Commercial targets for the system include media-monitoring firms and technology providers. Yet it could also be used by other sectors that monitor and analyse broadcast content. Standalone components – such as the Brand Detector and Genre Classification – are commercially available and the project partners have already signed first contracts.

Source: IST Results