Multimodal image fusion on ECG signals for congestive heart failure classification

The electrocardiogram, or ECG, records electrical signals from the heart to detect various heart conditions. It helps in the diagnosis of important cardiovascular diseases such as myocardial infarction (MI), cardiac arrhythmias, and congestive heart failure (CHF). Existing machine learning models and sophisticated deep learning techniques have primarily focused on using electrocardiogram (ECG) signals exclusively. However, to overcome the limitations of this approach, researchers have explored multimodal image-based fusion techniques, which have shown promising results, particularly when applied to real-time data such as ECG signals. In this work, two Multimodal Image-based Fusion techniques, Multimodal Image Fusion (MIF) and Multimodal Feature Fusion (MFF) are being used for the better classification of the ECG signals. The raw ECG data is first converted to create images of different modalities and then fused for classification. The different modality images formed are Gramian Angular Field (GAF), Markov Transition Field (MTF), and Recurrent Plot (RP) images. In the MIF method, the fused image of the three different modality images is provided to the convolutional neural network (CNN) for classification. When applying the MFF method, the features are first extracted from the obtained modality images by entering them in CNN and then these images are fused to get the required unique information. In this research, ECG5000 dataset is used for analysis. A newly designed model is used to classify the severity of Congestive Heart Failure (CHF) and attains accuracy using the proposed model is 97.79% for MIF and 98.19% for MFF methods respectively.