Fatigue assessment in bridges is crucial for maintaining the American road infrastructure. Typically, this involves measuring the bridge’s strain response under daily traffic loading. However, deploying and maintaining strain sensors is costly and labor-intensive compared to acceleration sensors. To address this issue, we propose a neural network architecture that can perform indirect sensing by estimating strain from measured acceleration response. Our proposed network employs convolutional neural networks (CNN) and transformers to account for uncertainties and noise associated with field data, and effectively convert accelerations to strain by capturing both the pseudo-static and dynamic features of the strain response. To demonstrate the effectiveness of our framework, we use field data collected from the Gene Hartzell Memorial Bridge in Easton, Pennsylvania, USA, as a case study. With our novel approach, we can estimate strain with high accuracy from acceleration data and reconstruct rainflow cycle counting diagrams that can subsequently be used for bridge condition and life cycle assessment.