TV programme bridges the semantic gap

Screenshot of Waisda.nl (currently offline, a new version will be launched in 2011)

Images for the Future

The tagging game Waisda? is one of a series of pilots in the Images for the Future project designed to explore the potential for public participation. Waisda? has already won several major prizes, in December Waisda? won the third prize at the Digital Heritage Conference 2010. Before the game already won the 'Best Archives on the Web Award' in the category Best Use of Crowdsourcing for Description. The game, which invites players to provide tags for moving images, has already produced some 400,000 tags. The successful video labelling game was developed by the Netherlands Institute of Sound and Vision in alliance with the Dutch public broadcasting organisation KRO, as part of the Images for the Future project. Tags can be useful in making it easier to search through online materials, as a look behind the scenes at Waisda? shows. Tagging specialist Lotte Belice Baltussen, who works on Waisda?, explained all about it.

You have won a lot of prizes; does that mean that you have a lot of players?
“There are approximately 600 registered players. That may not seem very many, but a total of more than 400,000 tags have already been added. Research has shown that there are also a number of very fanatical players. Around 250 people have played the game more than three times and 40 have even played it more than 10 times. These 400,000 tags provide us with an excellent dataset to work with.”

Why are the tags so important to you?
“They provide time-related information that makes it easier to search through programmes. In the past, television programmes were only described at the level of individual items. But if you are searching for a specific shot, you do not have very much to go on. You have to watch the whole segment of the programme until you find the right clip. Research conducted by a colleague shows that people want to be able to place orders quickly for individual clips. The tags offer media professionals a new way of searching much more effectively. Thanks to the tags, they can be sure of finding exactly the right shot. There are several ways to add time-based metadata. We use keyframes, a chronology of shots from a program that automatically generates intervals. The user can click on a shot and then jump immediately to the right point in the video. We are also working with the influx of subtitles on teletext page 888 and voice recognition for radio programs, both are also time-related metadata.”

How many people work on Waisda?
“The first version was developed by Johan Oomen and Maarten Brinkerink from the Netherlands Institute for Sound and Vision in alliance with the KRO. They did a lot of work with Just Vervaart, who was then employed by the KRO. I joined them in September of last year. During my degree course, I conducted research into tagging and this offered a good opportunity to continue that. We also work closely with the Q42 Internet agency and VU University Amsterdam. As part of the Presto PRIME EU project, the staff from the VU’s Web & Media group are conducting research into the tags. At the VU, around 4 to 5 people are working on Waisda? Q42 built the website.”

Is the game format an effective way of adding tags?
“Enabling greater access to moving images by means of tags is an extremely labour-intensive process. This is an area in which crowdsourcing really comes into its own. The more people who participate, the faster the process. The game format is also an enjoyable form of public interaction. The players discover that archives can actually be fun.”

How do you get people interested in the game?
“We do a lot of promotion on the website for the TV programme Boer zoekt vrouw (The farmer wants a wife) and use the element of competition to try to ensure that players come back for more. During the game, you can see whether other players are catching up with you, which encourages you to continue to play. We have also noticed that the awarding of prizes is very effective. During the period when it was possible to win the Boer zoekt vrouw board game, one player played for three hours at a single stretch and added more than 3300 tags. In the future, we intend to make even more use of the competition element.”

Is the archive material made accessible only of interest to media professionals?
“It is also of significant benefit to the general public, precisely because the tags have not been added by archivists or documentalists. Laypeople tend to use different words when describing and searching for materials. In the world of science, this is known as the ‘semantic gap’. The semantic gap is the difference between the specialist terms used by information professionals and the much freer search terms that laypeople use when they are searching. A game like Waisda?, which enables input from ordinary users, actually bridges this semantic gap. The tags added are a welcome addition to the terms that professionals add based on word lists.”

How good is the quality of the tags?
“The basic principle of Waisda? is that we assume that a tag is valid when two or more players add it within 10 seconds. This is called a match. Right now 40% of the tags has been matched. But if you look at all the tags players added, more than 90% in the database appear at least two times. It is expected that these tags still have to be matched. But there seems to be consensus overall."

What else have you learned from the evaluations?
“For the players, the game element takes precedence. Games normally have a clear beginning and end. People occasionally become confused because they find themselves in the middle of an episode of Boer zoekt vrouw or Barend & Van Dorp when they start playing. In the next version of Waisda?, to be launched in 2011, the games have a clearly-defined beginning and end. For this reason, we have chosen the Man bijt hond (Man bites dog) programmes, because all the episodes are broken up into segments. This means that you have a series of excerpts, each of which forms a game in itself. Of course, it is also enjoyable to watch. Man bijt hond has a lot of online fans. We hope that the visitors to the Man bijt hond website will also start playing Waisda? Waisda? will also feature more real game elements such as different levels and badges that can be won in order to make the game more challenging.”

When will it be possible to actually use the tags to search through programmes?
“We hope that that will be possible from the second quarter of 2011. The people at VU University Amsterdam have created a prototype search engine that works effectively. We also intend to add the tags to the information system provided by public broadcasting, along with the subtitles on teletext page 888 and the voice recognition for radio programmes. The subtitles are also time-related metadata. It will then be possible to search in three different layers: tags, voice recognition for audio materials and subtitles.”

Marie-José Klaver

Update December 2010: the website Waisda.nl is now offline as the pilot phase has reached its end. A new version of Waisda? will be launched in 2011 in alliance with public broadcasting organisation NCRV..