Lung diseases such as pneumonia, tuberculosis (TB), and COVID-19 are major global health concerns. Timely and accurate classification of these conditions from CT-CXR images is essential for effective diagnosis and treatment strategies. Single modality lacks the capability to capture the diverse features of various lung diseases. To overcome this limitation, Multimodal approaches integrate data from various sources to enhance disease understanding and classification accuracy. The proposed method utilizes a combination of Vision Transformer and Capsule Network i.e. VisCapsNet for feature extraction from CT-CXR images. It integrates features using a soft attentive dense weighted fusion (SAWF) model and SoftMax classifier for multi-class classification. In comparison with single modalities, multimodal fusion yields outstanding accuracies of 99.36% for binary, 99.1% for three-class, and 99.39% for four-class classification on CT-CXR datasets, underscoring its efficacy in accurately identifying diverse lung diseases from medical images, surpassing existing methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

VisCapsNet: Multimodality-Based Lung Disease Classification Using Deep Features of Integrated Vision Transformer with Capsule Network

  • Gautami Shingan,
  • Priya Ranjan

摘要

Lung diseases such as pneumonia, tuberculosis (TB), and COVID-19 are major global health concerns. Timely and accurate classification of these conditions from CT-CXR images is essential for effective diagnosis and treatment strategies. Single modality lacks the capability to capture the diverse features of various lung diseases. To overcome this limitation, Multimodal approaches integrate data from various sources to enhance disease understanding and classification accuracy. The proposed method utilizes a combination of Vision Transformer and Capsule Network i.e. VisCapsNet for feature extraction from CT-CXR images. It integrates features using a soft attentive dense weighted fusion (SAWF) model and SoftMax classifier for multi-class classification. In comparison with single modalities, multimodal fusion yields outstanding accuracies of 99.36% for binary, 99.1% for three-class, and 99.39% for four-class classification on CT-CXR datasets, underscoring its efficacy in accurately identifying diverse lung diseases from medical images, surpassing existing methods.