Scholarlink

Context <p>Adapting the User Interface (UI) of software systems to users’ requirements and their context of use is a challenging task. It involves determining the right adaptation, at the right time and place, to make it valuable for end-users. We believe that recent progress in Machine Learning (ML) techniques could provide useful ways in which to support adaptation more effectively. In particular, Reinforcement Learning (RL) has proven to be effective in planning a sequence of UI adaptations over a long time horizon. However, RL requires either manually specifying a reward function or learning a reward model. Currently there is no empirical evidence supporting the usefulness of reward models for UI adaptation.</p> Objective <p>This paper presents a confirmatory empirical study aimed at investigating the effectiveness of two different approaches to generating reward models in the context of UI adaptation using reinforcement learning: (1) a reward model derived exclusively from predictive Human-Computer Interaction (HCI) models (AUI-HCI), and (2) a reward model derived from predictive HCI models augmented by human feedback (AUI-HCI-HF), compared to non-adaptive (NA) interfaces.</p> Method <p>A controlled experiment with an AB/BA crossover design was conducted to evaluate the impact of these reward models on user experience, measured through objective and subjective engagement, as well as user satisfaction. Our study contributes to the understanding of how reward modeling can facilitate UI adaptation through RL.</p> Results <p>The results showed a significant improvement in objective engagement for AUI-HCI-HF compared to non-adaptive interfaces. However, no significant differences were found between AUI-HCI and non-adaptive interfaces for any of the other measurements, across any conditions.</p> Conclusion <p>Integrating human feedback into RL reward models enhances objective engagement, but its impact on subjective engagement and user satisfaction remains limited. While AUI-HCI-HF shows promise for improving interaction metrics, further research is needed to better align reward models with broader user perceptions and preferences, particularly compared to non-adaptive interfaces.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A comparative study on reward models for user interface adaptation with reinforcement learning

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年5月7日

查看全文
  • Daniel Gaspar-Figueiredo,
  • Marta Fernández-Diego,
  • Silvia Abrahão,
  • Emilio Insfran

摘要

Context

Adapting the User Interface (UI) of software systems to users’ requirements and their context of use is a challenging task. It involves determining the right adaptation, at the right time and place, to make it valuable for end-users. We believe that recent progress in Machine Learning (ML) techniques could provide useful ways in which to support adaptation more effectively. In particular, Reinforcement Learning (RL) has proven to be effective in planning a sequence of UI adaptations over a long time horizon. However, RL requires either manually specifying a reward function or learning a reward model. Currently there is no empirical evidence supporting the usefulness of reward models for UI adaptation.

Objective

This paper presents a confirmatory empirical study aimed at investigating the effectiveness of two different approaches to generating reward models in the context of UI adaptation using reinforcement learning: (1) a reward model derived exclusively from predictive Human-Computer Interaction (HCI) models (AUI-HCI), and (2) a reward model derived from predictive HCI models augmented by human feedback (AUI-HCI-HF), compared to non-adaptive (NA) interfaces.

Method

A controlled experiment with an AB/BA crossover design was conducted to evaluate the impact of these reward models on user experience, measured through objective and subjective engagement, as well as user satisfaction. Our study contributes to the understanding of how reward modeling can facilitate UI adaptation through RL.

Results

The results showed a significant improvement in objective engagement for AUI-HCI-HF compared to non-adaptive interfaces. However, no significant differences were found between AUI-HCI and non-adaptive interfaces for any of the other measurements, across any conditions.

Conclusion

Integrating human feedback into RL reward models enhances objective engagement, but its impact on subjective engagement and user satisfaction remains limited. While AUI-HCI-HF shows promise for improving interaction metrics, further research is needed to better align reward models with broader user perceptions and preferences, particularly compared to non-adaptive interfaces.