For decades, the Intelligence Quotient (IQ) has had numerous uses for humans but little importance when it comes to computers. With the focus and importance of AI research increasing, it was only a matter of time before we used this measure of intelligence to compare humans and machines.
Researchers from Microsoft and the University of Science and Technology of China built a deep learning machine that outperforms the average human on the types of problems that have always been toughest for computers, according to their study.
The test contains three categories of questions: logic questions (patterns in sequences of images); mathematical questions (patterns in sequences of numbers); and verbal reasoning questions (questions dealing with analogies, classifications, synonyms and antonyms). Computers have never been too successful at solving problems belonging to the final category, verbal reasoning, but the machine built for this study actually outperformed the average human on these questions.
In the past, computer scientists have used data mining techniques to analyze sets of texts to find links between words they contain and determine how those words relate to each other. This method has worked successfully for translating and other tasks, but it functions by assuming each word has a single meaning. Verbal tests as well as other tasks computer scientists are looking to accomplish with machines tend to focus on words with more than one meaning.
The researchers found it necessary to go beyond existing technologies to automatically solve verbal comprehension questions, so they created a framework consisting of three components.
The first element is a classifier that recognizes the specific type of a verbal question (Is it an analogy, classification, synonym or antonym problem?) The second prong involves leveraging a word embedding method that considers the multi-sense nature of words and the relational knowledge among words contained in dictionaries. Lastly, for each specific type of question, they developed a simple yet effective solver based on the obtained distributed word representations and relation representations. The overall result is a technique for recognizing the different meanings a word can have.
The researchers had the deep learning machine and 200 human subjects at Amazon’s Mechanical Turk crowdsourcing facility answer the same verbal questions. The result: their system performed better than the average human.
“Our RK model can reach the intelligence level between the people with the bachelor degrees and those with the master degrees, which also implies the great potential of the word embedding to comprehend human knowledge and form up certain intelligence,” the report states, based on the education levels of the human participants.
It looks like old Atari games aren’t the only thing computers are dominating these days.