Although hand gesture interaction is widely employed across a range of interactive systems, it remains comparatively underexplored within unified communication platforms (UCPs), particularly in the context of hybrid meeting environments that demand intuitive and seamless control and could benefit markedly from a spontaneous, touch-free control. Existing gesture vocabularies remain insufficiently aligned with the context of meetings and their functional controls. To address this lacuna, we conducted an elicitation study with 103 participants, each of whom proposed a gesture for eight core UCP commands. We report the resulting (824 proposals; 133 unique gestures), a four-dimensional taxonomy, an AI-based (LLM) classification, and agreement/dissimilarity analyses. Overall agreement was low ( \(\textrm{AR}=0.12\) ) yet peaked at 0.31 for the Ask-Question referent (46/103); the dissimilarity-consensus metric mirrored this pattern ( \(30.9\,\%\) versus \(\le 3.5\,\%\) for camera controls). Gestures were predominantly simple (one gesture per unit) ( \(91\,\%\) ) and dynamic ( \(75\,\%\) ), with the most common iconic ( \(38\,\%\) ) and symbolic ( \(27\,\%\) ) forms. Three LLMs assigned taxonomy labels with substantial inter-model agreement ( \(\kappa = 0.74\) –0.88), requiring manual tie-breaks for only \(0.015\,\%\) of the samples. We release the corpus and codes to enable reproducibility.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Eliciting User-Defined Mid-Air Hand Gestures for Hybrid Meeting Platform Control: Results, Insights, and Design Implications

  • Elizabete Munzlinger,
  • Fabricio Batista Narcizo,
  • Renata Briet,
  • Mario Tadashi Shimanuki,
  • Ted Vucurevich,
  • Dan Witzner Hansen

摘要

Although hand gesture interaction is widely employed across a range of interactive systems, it remains comparatively underexplored within unified communication platforms (UCPs), particularly in the context of hybrid meeting environments that demand intuitive and seamless control and could benefit markedly from a spontaneous, touch-free control. Existing gesture vocabularies remain insufficiently aligned with the context of meetings and their functional controls. To address this lacuna, we conducted an elicitation study with 103 participants, each of whom proposed a gesture for eight core UCP commands. We report the resulting (824 proposals; 133 unique gestures), a four-dimensional taxonomy, an AI-based (LLM) classification, and agreement/dissimilarity analyses. Overall agreement was low ( \(\textrm{AR}=0.12\) ) yet peaked at 0.31 for the Ask-Question referent (46/103); the dissimilarity-consensus metric mirrored this pattern ( \(30.9\,\%\) versus \(\le 3.5\,\%\) for camera controls). Gestures were predominantly simple (one gesture per unit) ( \(91\,\%\) ) and dynamic ( \(75\,\%\) ), with the most common iconic ( \(38\,\%\) ) and symbolic ( \(27\,\%\) ) forms. Three LLMs assigned taxonomy labels with substantial inter-model agreement ( \(\kappa = 0.74\) –0.88), requiring manual tie-breaks for only \(0.015\,\%\) of the samples. We release the corpus and codes to enable reproducibility.