What It Is
At this point, it is undeniable. Artificial-intelligence is on the rise.
Last year, Google’s DeepMind research team had a machine that can play arcade video games. Earlier this year, a Chinese researcher team demonstrated a face-recognition system that performs better than humans.
Now, Makarand Tapaswi and her team at the Karlsruhe Institute of Technology in Germany have put together a movie database that will serve as a testing ground for deep learning machines and their abilities to analyze stories. They believe that being able to answer questions about a story or movie is a good gauge of whether or not it was understood. So their goal is to create quizzes about movies that consist of some questions along with several possible answers, where only one is correct.
Two factors make these possible: First, we have a better understanding of many-layered neural networks and how to adjust them for specific tasks; second, the creation of the databases required to train these networks.
Most of the less-complex tasks are being taken on already—face recognition, object recognition, speech recognition, and so on. However, creating databases for more complex reasoning tasks, such as understanding stories, is considerably more difficult.
How Is It Done?
Their approach is straightforward. Tapaswi and her team begin by gathering plot synopses from Wikipedia for around 300 movies. Then they link this to the movie itself, which is a substantial body of data.
“An average movie is about two hours in length and has over 198K frames and almost 2,000 shots,” they say in the release. This means that there is a lot of data to sort through.
The team also gathers information from other databases. They then asked human annotators to read the synopses for each movie, and afterwards formulate a number of questions about each paragraph they read, along with the answer.
On average, the annotators wrote five questions per paragraph, and they also had to highlight a section of the text that provided the answer to each question. Finally, Tapaswi’s team asked the annotators to come up with a multiple choice quiz. The resulting database has over 7,000 questions about 300 films.
These questions are relatively straightforward for humans who have seen the corresponding film. But the machines do not do particularly well; still, the point is to prepare future machines that will hopefully perform better. For now, it is a small step forward towards creating machines that can understand stories and, by extension, us.
The database will be available online by the New Year here.