Mixed reality head-mounted displays (mxdR-HMD) have the ability to visualize volumetric medical imaging data in holograms to provide a full 3D perception of volumetric information. An effective user interface, however, has yet to be thoroughly studied. Tangible user interfaces (TUIs) enable interaction with holograms through objects that possess physical properties, which indicate their usage, and enable manipulation across multiple degrees of freedom. The main technological challenge in enabling TUIs on mxdR-HMDs, which are inherently limited in computational resources, is tracking the physical object in a timely and accurate manner and achieving seamless TUI integration. We introduce a markerless TUI employing a handheld planar object (PO) to interact with medical holograms, termed the mxdR hologram slicer (mxdR-HS). Our mxdR-HS enables users to slice the medical hologram to inspect regions of interest in real-time, integrate complementary data, and make annotations. We implemented our mxdR-HS on Microsoft HoloLens 1 – a computationally modest mxdR-HMD. We benchmarked our mxdR-HS against well-known tracking methods in various mxdR usage scenarios. Our mxdR-HS demonstrated a reduction in computational latency but with a marginal decline in tracking accuracy when compared to two marker-based methods. When compared to seven markerless tracking methods, our mxdR-HS improved in both latency and accuracy.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Mixed Reality Hologram Slicer (mxdR-HS): A Markerless Tangible User Interface for Interactive Holographic Medical Volume Visualization

  • Hoijoon Jung,
  • Younhyun Jung,
  • Michael Fulham,
  • Jinman Kim

摘要

Mixed reality head-mounted displays (mxdR-HMD) have the ability to visualize volumetric medical imaging data in holograms to provide a full 3D perception of volumetric information. An effective user interface, however, has yet to be thoroughly studied. Tangible user interfaces (TUIs) enable interaction with holograms through objects that possess physical properties, which indicate their usage, and enable manipulation across multiple degrees of freedom. The main technological challenge in enabling TUIs on mxdR-HMDs, which are inherently limited in computational resources, is tracking the physical object in a timely and accurate manner and achieving seamless TUI integration. We introduce a markerless TUI employing a handheld planar object (PO) to interact with medical holograms, termed the mxdR hologram slicer (mxdR-HS). Our mxdR-HS enables users to slice the medical hologram to inspect regions of interest in real-time, integrate complementary data, and make annotations. We implemented our mxdR-HS on Microsoft HoloLens 1 – a computationally modest mxdR-HMD. We benchmarked our mxdR-HS against well-known tracking methods in various mxdR usage scenarios. Our mxdR-HS demonstrated a reduction in computational latency but with a marginal decline in tracking accuracy when compared to two marker-based methods. When compared to seven markerless tracking methods, our mxdR-HS improved in both latency and accuracy.