Prof. Zhao Hai's Team Won the First Prize in the SQuAD 2.0 Challenge, the international top machine reading and understanding competition

Released Time: 2019-08-05

The natural language processing team, advised by Prof.Zhao Hai of the Key Laboratory of Intelligent Interaction and Cognitive Engineering, won the first prize for single model in the SQuAD 2.0 challenge on July 19, 2019. It was the first time that a single model exceeded human benchmark scores (including both EM and F1) , while the aggregative model exceeded 90% of the F1 benchmark for the first time. The main members of research team are Zhang Zhuosheng (master student) , Wu Yuwei (undergraduate student) , and Zhang Shuailiang (master student).




Machine reading comprehension is a key challenge in the field of AI, which aims to train machines to understand human natural language. SquAD (Stanford Question Answering Dataset, https://rajpurkar.github.io/SQuAD-explorer/) is regarded as the top competition in the field of machine reading comprehension. The goal is to let machines to read an article and then answer any question about it. SquAD 2.0 introduces unanswerable questions that require machines to make judgement when answering, which is closer to the human way of thinking in reading comprehension. Based on the recent research results in this field, the research team proposed several effective solutions to the problems in reading comprehension such as inaccurate semantic understanding and vulnerability to attack, which include using explicit semantic and syntactic knowledge to guide language model training. The performance of the reading comprehension system built by the team is now internationally leading.


In the history of the SquAD 2.0 challenge, only Google, IFLYTEK, the XLNet team and Pingan Technology have won the first prize. Thanks to the efforts of Prof. Zhao Hai’s team, Shanghai Jiao Tong University now becomes the fifth institute that enjoys this honor with the highest score among academic organizations (the next one is Seoul National University, ranking the 24th). At present the score (for single model) is higher than that of many renowned IT companies including Google, Facebook, Microsoft and Ali for all single/ aggregative models.


In addition, in 2019 Prof. Zhao’s team also outdid human in RACE, another machine reading comprehension flagship task and held the lead for half a year.


Machine reading comprehension technology can be applied in a lot of scenarios, including intelligent Q&A system, next generation search engine, dialogue robot system, etc.

Contact webmaster@cs.sjtu.edu.cn

Copyright @ 2013 SJTU Computer Science & Engineering All Rights Reserved