A Novel Fusion of CT/MRI and US Images Based on Depth Camera and Electromagnetic Tracking

This paper presents an automatic fusion imaging system that combines 3D CT (or MR) images with real-time ultrasound (US) acquisition using depth cameras and electromagnetic (EM) trackers. The system eliminates the need for physical markers and complex training, making image fusion accessible for physicians with varying experience levels. The fusion algorithm consists of two main parts: skin segmentation and rigid co-registration, both integrated into the US machine. The co-registration process aligns the surface extracted from CT images with the 3D surface acquired by a depth camera, facilitating rapid and effective fusion. Experimental tests in different settings confirm the system’s accuracy, computational efficiency, noise robustness, and operator independence. The co-registration error remains within acceptable ranges.