PrimARy: Intelligent System Based on Mixed Reality for Diagnosis and Assistance in Primary Care

This research aims to improve the diagnostic and support capabilities of healthcare professionals in primary care settings. We present PrimARy, a system that enables primary care workers to follow medical protocols in a guided manner using Mixed Reality and Artificial Intelligence. These protocols, defined using a node-based visual editor, can be automatically integrated into the Mixed Reality application, extending the system’s capabilities to different healthcare scenarios. The protocols can be enriched with documents and multimedia, and serve as the basis for the virtual assistant built into PrimARy to guide users in following a medical protocol. This functionality makes use of a Large Language Model deployed on a dedicated server for inference processes. As a practical application, the integration of a visual triage process to assess overweight and obesity is proposed. The ultimate goal is to scale our proposal for use in primary care centers, patients’ homes, and emergency situations by medical and nursing staff.