In the rapidly evolving landscape of network security and management, advanced machine learning techniques have become essential for analyzing and interpreting complex network traffic patterns. This research presents a classification of firewall actions using various transformer models such as Feature Tokenizer Transformer, Swin Transformer, and Vision Transformer. We present hybrid deep learning models, which combine artificial neural networks with transformers and convolutional neural networks integrated with transformers. We also present an ensemble learning model that combines Random Forest and gradient Boosting using a Voting Classifier. We use a dataset with 12 distinct features, including source and destination ports, NAT-translated ports, traffic volume, and session durations, to establish a solid foundation for classification tasks. Each model is applied to this dataset. Our experimental results demonstrate that the ensemble learning model achieves the highest accuracy of 99.81% for classifying the network traffic.