EVALUATING THE EXTENT OF ETHNIC BIASES IN FINBERT AND EXPLORING DEBIASING TECHNIQUES

Abstract: Language models are becoming increasingly popular. These models can contain social biases about various groups of people in them. The reproduction of biased beliefs can have harmful impacts on the groups they are about. We explore the extent of ethnic biases in the Finnish language model FinBERT. Our work focuses on biases about minority groups in Finland and we evaluate the extent of biases in the ethnic groups Roma, Finnish-Swedish, Sámi, Somali and Russian. In order to quantify the extent of biases, we use a template-based approach of calculating association scores between ethnicities and biased terms. We find that the model produces biased outcomes about the minority groups Roma and Somali. In order to mitigate the detected biases, we attempt debiasing FinBERT using dropout regularization and self-debiasing. The results of these two debiasing techniques do not produce satisfactory results and we conclude that debiasing ethnic biases and Finnish language models requires further research.