This study explores the applicability of explainable artificial intelligence (xAI) techniques in the analysis of deep learning models for anomaly detection in 5G/6G networks. With the increasing complexity of networks and network traffic, the mission to guarantee the security access points and devices against attacks and intrusions is also larger. Models used for these tasks operate like black boxes, making it difficult to understand and interpret their decisions at a human level. To address this challenge, we devised a case study with a real world dataset and a performant deep learning anomaly detection algorithm and implemented strategies to generate human understandable explanation through xAI algortihms. xAI can provide insights into the factors that lead to the detection of anomalies, allowing for greater transparency and reliability in the process. This work is part of the context of intelligent networks and is aligned with initiatives such as the Privateer project, contributing to the evolution of security in 5G/6G infrastructures. The integration of deep learning and xAI facilitates interaction between human operators and automated systems, promoting greater control over decision-making in modern networks.