Wireless Sensor Networks (WSNs) are increasingly used widely, which also increases the demand for effective Abnormal Detection (AD) methods to ensure the integrity and security of data. However, the inherent characteristics of WSN, such as limited resources (energy, processing, and memory), as well as dynamic operating environment, make abnormal detection more difficult. This article offers the idea of a new approaching detection for WSN combining Reinforcement Learning (RL) with AutoEncoder (AE) model. The objective of RL integration is to improve the performance of AE by optimizing parameters or selecting training data. The authors also consider changing batch size in the training process of RL agent to improve training efficiency and detection accuracy. As expected, compared to traditional methods, the proposed method will improve the detection rate and reduce the wrong alarm rate, either choose the best threshold for anomaly data.