The digitization of biological collections plays a crucial role in making scientific data more accessible and enabling long-term studies on biodiversity trends. While millions of herbarium specimens have been digitized worldwide, the extraction of metadata from handwritten labels remains a significant challenge. Traditionally, this task has been performed by experts or citizen scientists, ensuring high-quality data at the cost of substantial time and resources. In this paper, a system was developed that automates metadata extraction from herbarium specimen images. The system combines supervised deep learning techniques with a database-driven validation step and an open-source Large Language Model (LLM) to classify images of specific biological collections by collector, country, and year. It integrates an out-of-distribution detection mechanism using confidence scores, followed by a deep learning classification pipeline, and a database-supported LLM agent that refines the final predictions based on existing knowledge. The evaluation results demonstrate that a structured AI pipeline can streamline and improve the digitization of herbarium specimens by extracting metadata more efficiently than manual methods, all while preserving the level of accuracy required for scientific research.