Segmentation-Based Transformer Network for Automated Skin Disease Detection

Skin diseases encompass a wide range of conditions, from minor issues like acne and hives to severe ailments like eczema, psoriasis, and melanoma. Deep neural networks have been effective in classifying these disorders, but they often exhibit an ethnic gap, leading to lower detection accuracy for darker skin complexions. This paper demonstrates a novel approach making use of the vision transformer network by combining the original and binary segmentation masks using attention to minimize bias and classify skin diseases. The proposed methodology involves preprocessing images, generating segmentation masks, fine-tuning the vision transformer model, and evaluating performance using metrics like accuracy, precision, recall, and F1-score. The model achieved an accuracy rate of 91.3% when trained and tested on three different skin disease datasets. The experiment’s results indicate the proposed method’s effectiveness in classifying diseases of skin lesions belonging to the different ordinals of the Fitzpatrick scale and highlight the potential of further study in this field.