Attend All Options at Once: Full Context Input for Multi-choice Reading Comprehension

This paper proposes a method to capture the relations between options in Multiple-choice Machine Reading Comprehension (MMRC) tasks. MMRC is a form of question answering (QA) in which the question is about a given text, and multiple answers are provided as options. Capturing the relations between options is especially important for options with information references between them that cannot stand alone as responses to the questions, such as “None of the above”. Our method 1) takes the whole sample including identification of the passage, question, and all options as input for pre-trained language models, and 2) adds a fuser network to emphasize the information interaction between options. Experimental results show that our method improves over the common encoding approaches on COSMOS-QA, an MMRC dataset with between-option references, while having a relatively small impact on other MMRC datasets without references between the options. We conclude that our method actually helps to capture the necessary relationships between options. In addition, our method can reduce the memory usage required for training, and the model can be easily transferred to other domains and models.