Systematic Security Context Weighting for Trust Algorithms via AI/ML Model Performance Analysis
摘要
Malicious actors can introduce poisoned data by exploiting security vulnerabilities that may be present anywhere in networked systems. Depending on how the data is processed and used, this can have severe and potentially catastrophic effects. Using the data produced in such systems for the generation and evaluation of Artificial Intelligence (AI)/Machine Learning (ML) models, one may identify if malicious data is being introduced, though not necessarily where the intrusions occur. In this work we exploit the availability of metadata that is associated with the security context in which data is collected, transported and processed, as would be available, for example, in a data confidence fabric. We use this metadata in combination with iterative data weighting informed by that context. This facilitates the isolation of potential attack vectors or vulnerabilities by systematically training and evaluating low-impact AI/ML models while adjusting individual context weighting to discover the outlier features. With this approach, we can identify specific security or safety contexts of high significance based on anomalous impacts on model accuracy and loss evaluation computations.