Orthorectification for Dense Pixel-Level Spatial Calibration for Video-Based Structural Dynamics

Video-based structural dynamics techniques have shown great promise for applications such as monitoring the structural health of critical infrastructure such as locks and dams. Full-field approaches that make use of direct methods such as optical flow have the added attractive quality that they have demonstrated an ability to detect small damage on account of the high spatial density of pixels associated with imager measurements. For the case of inspecting critical infrastructure such as locks and dams, deformation measurements have also been shown to have utility. Imagers can be used to measure deformation; however, for the case of a complex 3D scene, every pixel can potentially have different sensitivity to deformation on account of the perspective transformation associated with pinhole camera photography. Telecentric lenses can be used to avoid perspective projection effects; however, telecentric lenses are large, expensive, and can only observe an area equal in size to their aperture greatly reducing their suitability for infrastructure inspection applications. In this work, we adapt techniques for orthorectification to attempt to address the issue of calibrating individual pixel deformations measurements across a scene. Orthorectification typically requires a height map information in the direction normal to the plane used to generate the orthophoto. The emergence of sensors such as time-of-flight imagers with high spatial resolution has made collecting these measurements more accessible for terrestrial infrastructure inspection applications. We demonstrate the ability to fuse subpixel motion measurements captured using a perspective camera, with 3D geometry data such as that which can be captured using a time-of-flight imager. This work focuses on the case of structures exhibiting planar geometry. We then show how these techniques impact the ability to perform video-based structural dynamics analysis.