Pomegranate crops are highly susceptible to bacterial, fungal, and viral diseases such as bacterial blight, wilt, anthracnose, and fruit rot, which adversely affect yield and quality. Traditional visual inspection methods are time-consuming and unreliable. To address this, an automated disease detection and classification system is proposed using deep learning and image processing. The system starts with image enhancement using CLAHE, noise removal, and resizing. An Attention U-Net model segments diseased regions from healthy fruit areas. These segments are then classified using a Vision Transformer (ViT), which captures complex features and long-range dependencies for accurate disease recognition. Grad-CAM is used for visualizing the decision-making areas of the model, enhancing transparency. The model is trained on a proprietary dataset containing multiple disease types and healthy samples. It is evaluated using accuracy, precision, recall, F1-score, and IoU. Designed for real-time field use, the model is deployable on mobile devices via Tensor Flow Lite or ONNX.