A Multimodal Document Viewer in Fully Immersive Virtual Reality

Immersive learning using fully immersive virtual reality (VR) is typically utilized to present 3D objects that cannot be represented easily on a computer screen. In the last few decades, the use of e-books to visualize documents has become prevalent. However, transferring an original document designed to be printed requires extraction of the document’s logical structure. The document must be broken down into fragments of text and images with the extraction of its logical structure. The fragments are then recomposed together to be presented as an e-book. In this paper, we propose a novel paradigm where documents can be presented in their original form without the need to be physically restructured, thanks to VR. With a VR headset, the proposed document visualizer allows users to access, display, and annotate documents. Such a system is particularly relevant for students and scholars in the humanities who investigate ancient documents where the distinct elements composing the document are intertwined. Furthermore, VR technology enhances the accessibility and equity of learning by offering various input modalities.