Scalable Real-Time Anomaly Detection in Financial Markets Using a Novel Machine Learning Model
摘要
Stakeholder objectives can be put at risk if anomalies are not detected sufficiently promptly in the constantly evolving financial systems. This paper recommends an innovative Machine Learning (ML) method, the Hybrid Temporal-Feature Embedding Model (HTEM), dedicated to real-time Anomaly Detection (AD). Fundamental to this framework is the use of a distributed system to store and process real-time data that has been collected through a strong Data Ingestion Layer. The HTEM framework is a vital component of this layout. The temporal flow of data collected from LSTM networks is precisely combined with the significance of individual features that are analyzed by autoencoders in this technique. By applying this dual embedding, a complete model of financial data sequences is accomplished. By applying a fully connected layer to provide this combined representation, the research affects an accurate anomaly value for each data point. If the overall rating is higher than a particular threshold, the data point is classified as an anomaly. Training this model frequently on known anomalies enables it to be fine-tuned to a higher level. The timely, accurate, and efficient finding of anomalies can be ensured through the incorporation of the HTEM within the model being proposed, protecting the interests of stakeholders in the financial market environment.