VRTumor: Integrating AI-Based Segmentation with Virtual Reality for Precise Tumor Analysis
摘要
Recently, many studies have demonstrated the effectiveness of technologies such as virtual reality (VR) and augmented reality (AR) in biomedical image analysis. However, these innovations have not been fully exploited to automate the process of cancer-level segmentation. Moreover, despite the immense potential of CT scan imagery in advancing research and clinical applications related to lung cancer, there is still a substantial need for sophisticated technological tools that can assist medical students, interns, and established physicians in achieving rigorous tumor analysis. To address this issue, we propose to integrate AI-driven image segmentation techniques with VR to achieve precise tumor analysis. The generation of data volumes is conducted using 3D reconstruction approaches. We utilize advanced segmentation techniques, including You Only Look Once version 8 (YOLOv8), Mask R-CNN, and U-Net to accurately delineate tumors. Furthermore, we apply a lung voxel-based reconstruction using iso-surface extraction, marching cubes, and data volume rendering. We generate a realistic display of lung tumor lesions in three-dimensional space through VR technology. Extensive computer simulations on CT image classification show improved efficiency compared to state-of-the-art methods, using a cancer dataset of 100 Algerian patients. Medical professionals have utilized the developed system to diagnose diseases. The results obtained illustrate the real-time effectiveness and precision of treatment plans.