AI and XR in Robotics

The liver is well known for its complex anatomy and high degree of variation in bile ducts and vessels, making liver surgery a challenging task. However, thanks to the rapidly evolving advances in computer technology and data science, technical innovations are developed to improve safety and precision of liver surgery. First of all, 3D models have found their way into preoperative planning of liver surgery to ensure safer resection planning. These models can be made semiautomatically or automatically with the help of artificial intelligence (AI). The surgeon can also consult these models during surgery to guide him/her through the resection plan. These approaches may result in shorter operating times, lower estimated blood loss, lower complications, and lower positive surgical margins. The 3D models can also be overlayed through augmented reality (AR) during the surgery, either on a separate display, head mounted display, or inside the robotic console. As such, it can be used for intraoperative guidance through targeting the lesion, identifying the relation of the tumor with the main anatomical structures, and defining resection margins. This is proven to be an added value in cases of vanishing lesions. However, a major drawback of these AR applications includes the lack of proper 3D model deformation and the time-consuming need for manual alignment and registration. Ideally, we need a flexible modeling process with dynamic registration capable of providing real-time feedback to the surgeon with minimal impact on surgical workflow. Another major AI innovation in the field of liver surgery is automated anatomical landmark identification, which can minimize the risk or even prevent injuries such as bile duct damage. Furthermore, automated surgical instrument tracking and logging of intraoperative events trough AI can assist in the evaluation of the surgical learning curve. If implemented correctly, AI landmarks can subsequently aid and facilitate AR 3D model registration.