In this paper, TextSAM-LoRA is presented as a novel pipeline for high-precision text detection that leverages the Segment Anything Model (SAM) as a base semantic segmentation model, efficiently fine-tuned with Low-Rank Adaptation (LoRA) layers for text segmentation tasks. Although SAM has demonstrated remarkable generalization capabilities across various segmentation tasks, its application to text detection remains largely underexplored. By integrating LoRA, SAM can be efficiently adapted to the text domain, significantly reducing both the computational cost and parameter overhead typically associated with fine-tuning large-scale models. The extensive experiments carried out on three benchmark datasets—CTW1500, MSRA-TD500 and Total-Text—demonstrate that TextSAM-LoRA achieves competitive state-of-the-art performance. Notably, on CTW1500 the approach outperforms previous methods with an F1-score of 90.4%, highlighting its exceptional ability to accurately detect complex curved texts. This work showcases the potential of combining foundational segmentation models with lightweight adaptation techniques for building specific and accurate text detection systems with limited computational resources.