Unsupervised multilingual distractor generation for fill-in-the-blank questions

Abstract: Fill-in-the-blank multiple choice questions (MCQs) play an important role in the educational field, but the manual generation of them is quite resource-consuming, so it has gradually turned into an attractive NLP task. Thereinto, question creation itself has become a mainstream NLP research topic, while distractor (wrong alternative) generation (DG) still remains out of the spotlight. Although several studies on distractor generation have been conducted in recent years, there is little previous work on languages other than English. The goal of this thesis is to generate multilingual distractors in Chinese, Arabic, German, and English across domains. The initial step is to construct small-sized multilingual scientific datasets (En, Zh, Ar, and De) and general datasets (Zh and Ar) from scratch. Considering that there are limited multilingual labelled datasets, unsupervised experiments based on WordNet, Word Embedding, transformer-based models, translation methods, and domain adaptation are conducted to generate their corresponding candidate distractors. Finally, the performance of methods is evaluated against our newly-created datasets, where three metrics are applied. Lastly, statistical results show that monolingual transformer-based together with translation-based methods outperform the rest of the approaches for multilingual datasets, except for German, which reaches its highest score only through the translation-based means, and distractor generation in English datasets is the simplest to implement, whereas it is the most difficult in Arabic datasets.