Multimodal Assistance System for the Care of Individuals in Early Stages of Dependency Using Augmented Reality and Artificial Intelligence

The present work explores the potential offered by the integration of various technologies such as augmented reality and artificial intelligence in the field of home-based care for dependent individuals. The proposed system focuses on providing assistance for everyday questions posed by individuals with certain levels of dependency or by new caregivers. This will start from the user’s interaction with a query about how to perform a specific action, accompanied by an image of their visual context and a manual detailing the routine, arrangement of the dependant’s belongings, and general information about the dependant. This will return a step-by-step guided response on what should be done, along with a list of objects to use that appear in the image of their environment. These objects will subsequently be highlighted in augmented reality for more comprehensive and understandable information. This solution aims to improve and prolong the independence at home of individuals in the early stages of dependency or dementia, as well as to enhance the adaptation of caregivers to new home environments, thereby improving the quality of life for patients and the work of caregivers.