In the last decade, artificial neural networks and deep learning techniques have revolutionized the field of medical imaging by improving image segmentation, detection, and classification. This advancement has dramatically improved diagnosis speed and treatment efficiency, especially when interpreting chest X-ray images to detect abnormalities within the thoracic cavity. To improve patient outcomes, lung diseases must be diagnosed early and accurately. For the analysis of chest X-ray images, convolutional neural networks have been used in several studies, and their performance in this regard is quite good. However, convolutional neural networks have been criticized recently for learning from non-relevant areas of an image, which may pose a problem regarding their reliability. This problem can be overcome by image segmentation and image slicing. Therefore, this research study proposes a novel lung segmentation-based approach for diagnosing and classifying lung disease that relies on a combination of medical and vision transformer models. The medical transformer model is used for lung area segmentation from chest X-ray, followed by a vision transformer model for classifying various lung diseases based on the segmented lung areas. As a result, the proposed approach achieved an average accuracy of 0.9229 and an average area under the curve of 0.7612 on the NIH chest X-ray dataset segmented lung areas. Accuracy and area under the curve are improved by 0.07% and 0.57%, respectively, demonstrating the effectiveness of the segmentation-based approach for lung disease classification.