XSynthMorph: Generative-Guided Deformation for Unsupervised Ill-Posed Volumetric Recovery

We introduce a novel unsupervised approach to recovering and registering a 3D volume from only two planar projections that exploits a previously-captured 3D volume of the patient. Such pre-capturing volume is readily available in many important medical procedures and previous methods already used such a volume. Earlier methods that work by deforming this volume to match the projections can fail when the number of projections is very low as the alignment becomes underconstrained. We show how to use a generative model of the volume structures to constrain the deformation and obtain a correct estimate. Moreover, our method is independant of the number, calibration and geometry of projections and could be adapted to new configurations without retraining. We evaluate our approach on a challenging dataset and show it outperforms state-of-the-art methods. As a result, our method could be used in treatment scenarios such as surgery and radiotherapy while drastically reducing patient radiation exposure.