<p>Structural health monitoring (SHM) is one of the most important solutions to the ongoing problem of deteriorating infrastructure. A crucial type of data needed for SHM is the displacement response of structures under loading, as these can be directly related to integrity and system performance, or can be used indirectly in analysis of the infrastructure. Video-based displacement measurement has become an increasingly accessible and viable approach to SHM. There is now an array of computer vision methods that extract displacement measurements using different components of the underlying video signal. However, all of these approaches are known to suffer from high levels of noise relative to conventional sensor installations. This study considers how to employ deep learning-based data fusion that combines measurements from several video analysis approaches to enhance the signal to noise ratio (SNR), resulting in an ensemble measurement approach. The emphasis of the study is on a convolutional neural network (CNN) architecture. A generative adversarial network (GAN) approach was also evaluated but was not able to reliably converge during model training. The CNN approach was evaluated using an experimental dataset of vibrations from a lab-scaled structural system with varying dynamic properties. The results show that the CNN fusion approach improves the loss function significantly and serves to effectively denoise the individual signals extracted from the video.</p>