The Internet’s vault of how-to information isn’t just useful for humans these days.
A European project called RoboHow is teaching machines to learn to complete tasks from instructions on the web. One German robot called PR2 learned to cook pizza and pancakes by reading WikiHow and watching YouTube videos.
The researchers behind the four-year project see exploring ways to teach robots to understand language as the route to achieving their goal. They aim to enable machines to perform everyday human-scale activities as competently as humans. Put simply, they want to be able to tell a robot what to do rather than needing to program it to perform each precise movement.
SEE ALSO: This Cheap iPhone Attachment Lets Criminals Steal Your Bank PIN In Seconds
PR2’s cooking lessons began with some time reading WikiHow, according to a video on the project. At this point, the robot was able to load basic instructional information into its knowledge base. This is a visual representation of what PR2 was able to understand about making pancakes from reading a how-to article:
The basics are there, but much of the important information must be inferred—this is where the robot’s need to visually watch someone make pancakes comes into play. The researchers showed PR2 instructional videos of someone making pancakes, as well as human demonstrations. They also performed kinesthetic teaching, where they physically moved the robot to show it how to perform the nuanced motions like sliding the spatula under the pancake, raising it and flipping it.
One day, reading the instructions alone may be enough, though.
Teaching robots to “read” natural language in the way that humans do (so they’re able to understand and interpret context and verbal reasoning, not just deal with words or phrases) is among the highest priorities of artificial intelligence researchers. This has been one of their biggest challenges thus far, but breakthroughs are happening.
Google’s AI firm, for example, recently discovered they could improve their machines’ ability to understand natural language with articles by CNN and The Daily Mail. They’ve been in search of a massive database of annotated natural language, and they struck gold when they realized the news sites’ articles include bullet point summaries. And since there are hundreds of thousands of those articles, they made for a selection of annotated natural language vast enough for the neural network to learn from.
This news isn’t far off from the recent announcement that researchers from Microsoft and the University of Science and Technology of China built a deep learning machine that outperforms the average human on verbal reasoning questions on the IQ test. Computers have never been too successful at solving problems that deal with analogies, classifications, synonyms, antonyms and multiple word meanings, but that’s changed.
(h/t MIT Technology Review)
