To enhance the ability to detect anomalous activities in online manner, streaming anomaly detection often faces the challenge of adapting to changing data trends. Although existing methods for streaming anomaly detection exhibit significant performance, most of them are limited by their inability to fully account for long-term memory of global data distribution, the lack of rapid adaptation to local data distribution changes, and the low generalizability of manually defined thresholds. To address these challenges, we propose an adaptive anomaly detection framework designed for streaming data, named AdpStream. Specifically, AdpStream first utilizes a feature extraction module to encode streaming data, capturing correlations among variables to generate embeddings for each data record. Secondly, a global memory module is used to store embeddings from long historical windows, preserving diverse global patterns of normal data with dynamic update strategy. The local adapter module calculates reconstruction loss for each record and determines its anomaly probability using local information. Finally, the anomaly criterion module assesses anomaly levels by combining the reconstruction error of the current record with its similarity to data in the global memory. Extensive experiments on 10 real-world datasets are conducted to demonstrate the superior performance of AdpStream. The source code is available at: https://anonymous.4open.science/r/AdpStream-2C2B .