Evaluating the robustness and readiness of large frontier models in health AI applications
摘要
Large frontier models such as GPT-5 and Gemini have demonstrated remarkable performance in a wide range of health application benchmarks. However, underneath the seemingly promising results lie salient growth areas, especially in cutting-edge frontiers such as multimodal reasoning. Here we systematically apply and integrate a series of adversarial stress tests to assess the robustness of flagship models and health benchmarks. Our study reveals prevalent brittleness in the presence of simple adversarial transformations: leading systems can guess the correct answer even with key inputs removed yet may get confused by the slightest prompt alterations while fabricating convincing but flawed reasoning traces. Using clinician-guided rubrics, we demonstrate that popular health benchmarks vary widely in what they truly measure. Our study reveals considerable gaps between benchmark performance and the robustness evidence needed to support claims about multimodal medical reasoning in health applications.