Aquaculture production significantly influences overall fish production, yet it is often adversely affected by various fish diseases. These diseases can be effectively identified by analyzing the condition of the fish’s skin. Consequently, there is a growing demand for automated fish skin disease detection methods. By implementing such automated approaches, the efficiency and accuracy of disease detection can be enhanced, leading to better management of fish health and, ultimately, more sustainable aquaculture practices. In this work, we propose a novel Transformer based modified YOLO approach for detection of five different fish skin diseases. We propose a Transformer feature extraction module (TFEM) to effectively capture the long-range dependencies from input image. The proposed TFEM is incorporated in the YOLOv7 backbone for efficient feature learning. We assessed the performance of our proposed TFEM by comparing it with various YOLOvX approaches to confirm its effectiveness. Both qualitative and quantitative results demonstrate that our method is highly capable of accurately detecting five distinct fish diseases. The source code is available at: https://github.com/shrutiphutke/Fish_disease_detection_YOLO_transformer