Structural health monitoring aims to develop robust methods to determine whether a structure’s behavior remains within normal limits. However, operational and environmental variability can significantly affect a structure’s dynamic response. Neglecting these influences may lead to false negative indications when a damaged structure is deemed undamaged or false positive indications when an undamaged structure is labeled as damaged. It is therefore essential to monitor the environmental variables and understand their effect on the structure. This paper presents a long-term study of the Hell Bridge Test Arena, a steel riveted truss bridge used as a full-scale damage detection test structure. The natural frequencies of the bridge are obtained through automatic operational modal analysis and modal tracking over an extended period, including one year in the undamaged conditions and a few months in damaged conditions. A two-component Gaussian Mixture Model (GMM) that partitions the data based on temperature was employed. The GMM was implemented with a modified covariance estimation approach to address missing frequency entries. The study examines how environmental variables influence the bridge’s natural frequencies under both damaged and undamaged conditions. The results indicate that the combination of modal tracking and a modified GMM is generally capable of separating damaged from undamaged samples demonstrating the potential of enhancing this integrated approach for robust damage detection in civil infrastructure.