Deep learning models have emerged as a powerful and cost-effective tool for medical image classification tasks, particularly for oncology. Lung CT computer-aided diagnosis system has been one of the most outstanding tasks with superior performance. However, Lung CT-based CAD systems are vulnerable to adversarial attacks, which could lead to misdiagnosis in clinical practice. Adversarial training, an effective conventional defense method, has proven to be useful in protecting against adversarial attacks and increasing the robustness of the model. However, adversarial training is costly, which requires additional training on a large number of samples. Instead, we developed feature fusion, a defense method with feature enhancement that does not require additional training while significantly improve the robustness of the model. Feature fusion is also generalizable to multiple model architectures (VGG16, ResNet50, and Vision Transformer Base-16) with superior performance. The results show a significant reduction in the performance drop for three first-order gradient adversarial attacks, fast gradient sign method (FGSM), basic iterative method (BIM), and projected gradient descent (PGD). In addition, incorporating additional adversarial training after pretraining with the feature fusion-based classification method can further significantly strengthen the robustness of the model.