Person-Centric Sensing in Indoor Environments

Person-centric sensing within indoor environments has garnered substantial interest due to its capacity for enabling automation across a wide array of domains, including smart homes, active assisted living, security, and surveillance. Presently, optical modalities, such as RGB, depth, and thermal imaging, enjoy widespread adoption in the realm of Human Activity Recognition (HAR). This adoption can be attributed to the effectiveness of deep learning algorithms and the extensive availability of publicly accessible image datasets. Nevertheless, unconventional modalities, such as radar, WiFi, seismic, and environmental sensors, are emerging as viable alternatives due to their capability for contactless long-range sensing in spatially constrained environments and their ability to preserve visual privacy. The preservation of visual privacy is recognized as a key requirement for user acceptance in privacy-sensitive applications. Despite the demonstrated potential of these emerging modalities, their widespread adoption has yet to be realized. This can be partly attributed to persistent challenges, including the unavailability of off-the-shelf sensing hardware, lack of publicly available training data, complexities associated with multi-modal data processing, and poor modal generalization to new environments. This chapter aims to shine a light on these open problems and provides a comprehensive examination of the landscape of modalities in person-centric sensing in indoor environments, encompassing both well-established and emerging modalities.