With the rapid development of the electric vehicle (EV) industry and the fast-paced construction of digital infrastructure for smart grids, the number of electric vehicles is increasing daily. The EV charging process requires coordination among the grid, charging piles, batteries, and vehicle systems, resulting in a large amount of multivariate time series data. Effectively utilizing the charging data collected by smart grids to ensure the safety of the EV charging process has become a new challenge. Traditional methods mainly focus on detecting individual devices and do not effectively utilize the diverse data from the entire charging chain. To address this issue, this paper proposes a deep learning-based anomaly detection technology for EV charging. This method constructs a multi-scale time window signature matrix to deeply mine data features and combines it with a Transformer model to establish an anomaly detection model. Experiments show that this method can effectively identify charging anomalies, enhancing the ability of smart grids to recognize and detect abnormal EV charging behaviors.