MRC Examples Answerable by BERT without a Question Are Less Effective in MRC Model Training

Abstract

Models developed for Machine Reading Comprehension (MRC) are asked to predict an answer from a question and its related context. However, there exist cases that can be correctly answered by an MRC model using BERT, where only the context is provided without including the question. In this paper, these types of examples are referred to as easy to answer'', while others are ashard to answer'', i.e., unanswerable by an MRC model using BERT without being provided the question. Based on classifying examples as answerable or unanswerable by BERT without the given question, we propose a method based on BERT that splits the training examples from the MRC dataset SQuAD1.1 into those that are easy to answer'' orhard to answer''. Experimental evaluation from a comparison of two models, one trained only with easy to answer'' examples and the other withhard to answer'' examples demonstrates that the latter outperforms the former.