Approaches based on language models for aspect extraction for sentiment analysis in the Portuguese language

This work addresses the gap in aspect extraction techniques for Portuguese by adapting methods originally designed for English. It focuses on TV devices and literary reviews in the TV and ReLi datasets. For this, models based on the BERT architecture were employed, including pre-trained general domain (BERTimbau) and specific domain models (BERTtv and BERTreli). Also, this paper contributes with a novel double embedding technique that merges these models. We further explored the potential of large language models (LLMs) with a Portuguese-trained LLaMa variant, Cabrita. Efficient fine-tuning techniques such as LoRA (low-rank adaptation) for BERTimbau and QLoRA (quantized low-rank adaptation) for Cabrita were applied to optimize resource demands. The BERTimbau model, adjusted with LoRA, achieved the highest F1 scores (0.846 for TV and 0.615 for ReLi), while Cabrita showed lower performance (0.68 for TV and 0.46 for ReLi). This study underscores the potential of adapting and optimizing existing techniques for aspect extraction in Portuguese, marking a significant advancement in the field