Towards Enhancing Extended Reality for Healthcare Applications with Machine Learning

Extended Reality is expanding in healthcare with regard to the quality of vision the operator experiences. On the other hand, machine learning has provided numerous capabilities to enhance user interaction. This paper focuses on using the visualizations of extended reality (XR) methods alongside functionalities provided by intelligent systems. We propose three hypothetical methods for integrating the two fields to provide healthcare solutions. The three use cases of treatment visualization, therapeutic strategies and decision-making are proposed to enhance healthcare. The treatment visualization case uses image segmentation and virtual reality (VR). We segment the medical image that the medical worker visualizes through the VR headset. We propose an adaptive mixed reality system that can change according to the user’s mood for appropriate therapy. Neural models recognize the user’s emotions and adapt the system accordingly. An augmented reality chatbot has been proposed to influence timely decision-making during medical situations. Each case study uses different tools from the other and focuses on a specific healthcare-related solution.