Advancing Trustworthy AI: A Comparative Evaluation of AI Robustness Toolboxes
摘要
Increasing reliance on Artificial Intelligence (AI) across diverse sectors, from healthcare to telecommunications, necessitates robust systems capable of withstanding adversarial attacks and unforeseen inputs. This study aims to evaluate and fortify AI robustness through comprehensive toolboxes that offer pre-built attack and defense methods across various data types, such as image, text, and audio. This paper examines eleven AI robustness toolboxes, analysing their features, attack methods, strengths, and limitations. It compares these toolboxes on fourteen attributes based on community support, popularity, and versatility, employing a novel scoring rubric. The analysis identifies six areas for future research and guides researchers and developers in selecting the most appropriate tool for their specific needs. This study seeks to improve the practical usability of existing toolboxes and advance AI robustness assessment and benchmarking methodologies by identifying their limitations and potential gaps. This work serves as a foundational review of AI robustness toolboxes, empowering responsible AI development and contributing to building secure and trustworthy systems.