Multimodal and multi-time-point fusion approach for automated diagnosis and grading of carotid atherosclerosis using bilateral ultrasound images and metadata
摘要
Carotid atherosclerosis is a common vascular disease characterized by lipid deposition and plaque formation within the carotid artery walls, which may lead to restricted blood flow and even severe complications such as stroke. This study aims to develop an automated evaluation system based on the ResNet and transformer framework to diagnose carotid atherosclerosis and grade its severity using bilateral ultrasound images of the neck at different time points along with related metadata. Given the complexity and multifactorial nature of carotid atherosclerosis, we designed a cross-time-point image fusion strategy to capture the dynamic changes and spatial distribution characteristics of lesions. Additionally, a masked cross-time-point attention mechanism was employed to optimize information interaction between images at different time points, reducing noise interference. Furthermore, a multilayer perceptron was utilized to extract metadata features, further enhancing the model’s expressive capabilities. Extensive comparative and ablation experiments conducted on a self-constructed dataset demonstrated the effectiveness and robustness of the proposed method, providing new technical support for the early screening and precise treatment of carotid atherosclerosis.