Metadata bring order to digital chaos

September 10, 2008

MP3 files, video streams, digital images – the flood of multimedia data swells higher every day. New systems help the user to keep tabs on it all. At the International Broadcasting Convention IBC in Amsterdam on September 12 through 16, Fraunhofer scientists will present professional solutions for the intelligent searching, analysis and administration of multimedia data.

Every day new songs appear in online music stores, new digital images in virtual photo albums, new videos in portals such as YouTube – how can you get to grips with this flood of data? How do you find the songs, pictures or videos you are looking for?

Researchers at the Fraunhofer Institute for Digital Media Technology IDMT in Ilmenau have developed three technologies that help to classify media content, launch effective searches, and organize databases. In-depth analysis of the audio and video data can provide more information about the properties of the appropriate starting material – information known as the “metadata” – than is possible when the material is manually cataloged by experts.

Are you looking for some jazz music, preferably not too fast and with a saxophone solo? That’s no problem for the “Digital Music Finder”. The software manages your music collection and searches through it with the aid of content-based metadata. Complex properties such as the genre or the segmentation of a song into verse, refrain and soli can be included in the search – regardless whether it is carried out in your own archive or in an online music store. It is possible thanks to various technologies that are based in part on the MPEG 7 standard: With their help, the files are automatically supplemented by semantic descriptions. Besides specifying the title and the artist, these might include details about the tempo, the melody or the structure of the piece. But in addition to retrieving the desired music titles, the Digital Music Finder identifies similar-sounding songs and compiles personalized music recommendations for the user. “The technology is a powerful solution for organizing music archives, compiling playlists or planning programs – both for private users and for radio stations,” says Holger Großmann, head of the metadata department at the Fraunhofer IDMT.

Meanwhile, the technology can also be used to search through digital photo archives. This means that it is now possible to have photo collections on certain themes compiled automatically. In combination with a playlist that is also compiled automatically to harmonize with the mood of the photographs, completely new applications can be created.

Another new technology can distinguish between voice and music. It indicates the periods of time during a radio program or a podcast in which someone is speaking or music is being played. The advantage is that, during TV or radio broadcasts, the program’s musical content can be automatically determined and used for statistical evaluations. What is more, the technology can immediately locate certain pieces of music or parts of the moderation. The system sensitivity can be adapted to various applications, making it possible even to filter out music segments that have voice signals superimposed on them.

The “VideoID Manager” tidies up video collections. The system identifies video material and checks it for any copyrights that may exist. To do this, it takes a “digital fingerprint” of the unknown video. Various features such as the color distribution, optical flow and color histogram are extracted and classified. The sequence is compared with the entire video material held in a database and thus identified. The recognition system is so robust that it remains largely unaffected by data compression loss or signal distortion. This method enables hitherto unidentified data to be recognized and then supplemented by the respective metadata. Possible uses of the “VideoID Manager” might be in monitoring and analyzing videos, TV and Web articles, or for monitoring music videos and commercials to check how frequently these are broadcasted.

Provided by Fraunhofer-Gesellschaft