A Multimodal and Multiresolution Fusion Using Convolutional Neural Networks
摘要
Recent advances in circuit design and sensing technology have made it increasingly feasible to collect data from multiple sensors simultaneously, enabling more accurate and comprehensive inferences in biomedical applications. This chapter presents a novel convolutional neural network (CNN) architecture designed specifically for the fusion of multimodal and multiresolution sensor data—signals acquired at differing sampling frequencies. Unlike conventional approaches that rely on signal padding or resampling to align resolutions, this model directly integrates disparate inputs, supporting high resolution temporal inference without the need for frequency normalization. The model’s performance is evaluated for sleep apnea event detection, by fusing three different sensor signals, namely, electrocardiogram (ECG), peripheral oxygen saturation signal (SpO2), and abdominal movement signal, obtained from UCD St. Vincent University Hospital’s sleep apnea database. Results show that fusion using all three signals yields an accuracy of 99.72% and a sensitivity of 98.98%. The chapter also introduces a selective dropout mechanism to prevent overfitting to any single high-resolution input and enhance model robustness in the presence of corrupted or missing data. Furthermore, the model exhibits incremental performance gains as more sensor inputs are added, demonstrating its scalability and generalizability. Model pruning techniques for complexity reduction are also explored, highlighting the potential for real-time deployment in resource-constrained environments.