GeoCQA: A Large-Scale Geography-Domain Chinese Question Answering Dataset from Examination

We present GeoCQA, the largest multiple-choice Chinese Question answering dataset in the geographic domain, evaluating the high-level reading ability of logic reasoning and prior geographic domain knowledge integration of a question answering (QA) model. GeoCQA contains 58,940 questions from real-world scenarios and has been collected from the high school geography examination which aims to evaluate students’ mastery of the geographic concept and their ability to use geographic knowledge to solve problems. To investigate the challenges of GeoCQA to existing methods, we implement both rule-based and best neural methods and find that the current best method can achieve 71.90% of test accuracy, while unskilled humans and skilled humans can reach 80% and 96% accuracy respectively, which shows that GeoCQA is challenging to the current methods and the performance still has space to improve. We will release GeoCQA and our baselines to bring more data sources to the community and hope that it can help to promote much stronger Chinese QA models in the future (https://github.com/db12138/GeoCQA ).