BERT Downstream Task Analysis: Named Entity Recognition in Serbian

This paper presents a comparison of different architectures and techniques for preparing named entity recognition (NER) models for the Serbian language via integrating BERT with spaCy. Models were trained to recognize seven different named entity types (persons, locations, organisations, professions, events, demonyms, and artworks), and are trained on the dataset containing Serbian novels published between 1840 and 1920, publicly available newspaper articles and sentences generated from the Wikidata knowledge base and Leximirka lexical database. We explore various configurations and several training pipelines that differ in complexity and functionality. Some are dedicated solely to NER, while others encompass additional features like Part-of-speech tagging and lemmatization. One of the key aspects of this work involves testing different versions of BERT, with varied architectures, sizes, and pre-training corpora that contain the Serbian language. This approach allows us to evaluate the trade-offs between model complexity and performance and offers a nuanced understanding of how different configurations impact the efficiency and effectiveness of NER task in Serbian.