Exploring Cross-Lingual Transfer Learning for Swedish Named Entity Recognition : Fine-tuning of English and Multilingual Pre-trained Models

Abstract: Named Entity Recognition (NER) is a critical task in Natural Language Processing (NLP), and recent advancements in language model pre-training have significantly improved its performance. However, this improvement is not universally applicable due to a lack of large pre-training datasets or computational budget for smaller languages. This study explores the viability of fine-tuning an English and a multilingual model on a Swedish NER task, compared to a model trained solely on Swedish. Our methods involved training these models and measuring their performance using the F1-score metric. Despite fine-tuning, the Swedish model outperformed both the English and multilingual models by 3.0 and 9.0 percentage points, respectively. The performance gap between the English and Swedish models during fine-tuning decreased from 19.8 to 9.0 percentage points. This suggests that while the Swedish model achieved the best performance, fine-tuning can substantially enhance the performance of English and multilingual models for Swedish NER tasks.