Telepresence technologies are increasingly essential for effective remote collaboration and interaction across various sectors, including industry, healthcare, and education. Traditional telepresence systems predominantly rely on conventional video capture, offering limited immersion despite adequate resolution for detailed visual tasks. Although 360-degree video provides enhanced immersion through panoramic views, practical constraints such as insufficient resolution and streaming bitrate significantly degrade detailed visual inspection capabilities. To overcome these limitations, we propose a modular hybrid telepresence system that integrates immersive 360-degree video with targeted high-resolution imagery captured by standard cameras. Implemented on a mobile telepresence robot (Double 3) and displayed within a portable Cave Automatic Virtual Environment (CAVE)-like system named VDen, our solution features a modular image registration framework capable of switching between multiple feature-matching algorithms (SIFT, LoFTR, LightGlue+DISK). Experimental results validate our system’s capability to effectively balance immersive panoramic perception and detailed visual inspection, outperforming traditional telepresence and purely super-resolution-based approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Modular Hybrid Telepresence System Integrating Immersive 360-Degree Video with On-Demand High-Resolution Imaging

  • Jiapeng Chi,
  • Carsten Neumann,
  • Carolina Cruz-Neira,
  • Dirk Reiners

摘要

Telepresence technologies are increasingly essential for effective remote collaboration and interaction across various sectors, including industry, healthcare, and education. Traditional telepresence systems predominantly rely on conventional video capture, offering limited immersion despite adequate resolution for detailed visual tasks. Although 360-degree video provides enhanced immersion through panoramic views, practical constraints such as insufficient resolution and streaming bitrate significantly degrade detailed visual inspection capabilities. To overcome these limitations, we propose a modular hybrid telepresence system that integrates immersive 360-degree video with targeted high-resolution imagery captured by standard cameras. Implemented on a mobile telepresence robot (Double 3) and displayed within a portable Cave Automatic Virtual Environment (CAVE)-like system named VDen, our solution features a modular image registration framework capable of switching between multiple feature-matching algorithms (SIFT, LoFTR, LightGlue+DISK). Experimental results validate our system’s capability to effectively balance immersive panoramic perception and detailed visual inspection, outperforming traditional telepresence and purely super-resolution-based approaches.