Mixed Reality Hologram Slicer (mxdR-HS): A Markerless Tangible User Interface for Interactive Holographic Medical Volume Visualization
摘要
Mixed reality head-mounted displays (mxdR-HMD) have the ability to visualize volumetric medical imaging data in holograms to provide a full 3D perception of volumetric information. An effective user interface, however, has yet to be thoroughly studied. Tangible user interfaces (TUIs) enable interaction with holograms through objects that possess physical properties, which indicate their usage, and enable manipulation across multiple degrees of freedom. The main technological challenge in enabling TUIs on mxdR-HMDs, which are inherently limited in computational resources, is tracking the physical object in a timely and accurate manner and achieving seamless TUI integration. We introduce a markerless TUI employing a handheld planar object (PO) to interact with medical holograms, termed the mxdR hologram slicer (mxdR-HS). Our mxdR-HS enables users to slice the medical hologram to inspect regions of interest in real-time, integrate complementary data, and make annotations. We implemented our mxdR-HS on Microsoft HoloLens 1 – a computationally modest mxdR-HMD. We benchmarked our mxdR-HS against well-known tracking methods in various mxdR usage scenarios. Our mxdR-HS demonstrated a reduction in computational latency but with a marginal decline in tracking accuracy when compared to two marker-based methods. When compared to seven markerless tracking methods, our mxdR-HS improved in both latency and accuracy.