Real-Time Device Detection with Rotated Bounding Boxes and Its Clinical Application

Interventional devices and insertable imaging devices such as transesophageal echo (TOE) probes are routinely used in minimally invasive cardiovascular procedures. Detecting their positions and orientations in X-ray fluoroscopic images is important for many clinical applications. Nearly all interventional devices used in cardiovascular procedures contain a wire or wires and are inserted into major blood vessels. In this paper, novel attention mechanisms were designed to guide a convolution neural network (CNN) model to the areas of wires in X-ray images. The first attention mechanism was achieved by using multi-scale Gaussian derivative filters in the first convolutional layer inside the proposed CNN backbone. By combining these multi-scale Gaussian derivative filters together, they can provide a global attention on the wire-like or tube-like structures. Furthermore, the dot-product based attention layer was used to calculate the similarity between the random filter output and the output from the Gaussian derivative filters, which further enhances the attention on the wire-like or tube-like structures. By using both attention mechanisms, a high-performance CNN backbone was created, and it can be plugged into light-weighted CNN models for multiple object detection. An accuracy of 0.88 ± 0.04 was achieved for detecting an echo probe in X-ray images at 58 FPS, which was measured by intersection-over-union (IoU). Based on the detected pose of the echo probe, 3D echo can be fused with live X-ray images to provide a hybrid guidance solution. Codes are available at https://github.com/YingLiangMa/AttWire .