Mixed Reality (MR), which overlays virtual user interfaces (UIs) onto physical environments, necessitates UI adaptation to complex scenes and tasks. While prior work focused on adaptive spatial UI layout in MR, functional adaptation such as widget recommendation remains underexplored. We present a user study ( \(n = 16\) ) using a Large Language Models (LLMs)-powered widget recommender system (RS) as a technology probe to investigate how context-aware recommendations affect user experience. The system uses LLMs with contextual data (reading text, video transcript and typed data) to suggest MR widgets. Results show that widget recommendations facilitated access to context-relevant functionalities and simplified task workflows, thus enhancing the user experience and reducing workload. However, the usability of the widget RS depends on appropriate widget design and recommendation strategies that enable personal customization. This study serves as an initial step toward MR widget RSs and offers insights for adaptive user-RS interactions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Exploring the Design of Context-Aware Widget Recommender System in Mixed Reality

  • Yiming Sun,
  • Yang Zhan,
  • Tatsuo Nakajima

摘要

Mixed Reality (MR), which overlays virtual user interfaces (UIs) onto physical environments, necessitates UI adaptation to complex scenes and tasks. While prior work focused on adaptive spatial UI layout in MR, functional adaptation such as widget recommendation remains underexplored. We present a user study ( \(n = 16\) ) using a Large Language Models (LLMs)-powered widget recommender system (RS) as a technology probe to investigate how context-aware recommendations affect user experience. The system uses LLMs with contextual data (reading text, video transcript and typed data) to suggest MR widgets. Results show that widget recommendations facilitated access to context-relevant functionalities and simplified task workflows, thus enhancing the user experience and reducing workload. However, the usability of the widget RS depends on appropriate widget design and recommendation strategies that enable personal customization. This study serves as an initial step toward MR widget RSs and offers insights for adaptive user-RS interactions.