Autonomous cyber defence trains intelligent blue agents that can protect systems independently, without human oversight. Recent research in this field often uses deep reinforcement learning to train and evaluate blue agents against simulated cyber attacks. However, most training relies on static, hardcoded red agents, which may fail to reflect real-world adversarial strategies. Moreover, deep reinforcement learning policies are potentially vulnerable to adversarial perturbations, raising the following question: can attackers exploit these vulnerabilities by choosing actions that cause the blue agent to make mistakes, allowing the attacker to evade detection? In this work, we use the CybORG environment to show that a carefully designed adversarial policy can substantially degrade the performance of a blue agent trained on hardcoded red agents. Our results indicate that, while the blue agent successfully contains the original red agents, its ability to protect a target operational server drops by over 40% when pitted against the adversarial agent. These findings confirm the effectiveness of adversarial policies against blue agents, and highlight the need for robust training methodologies to ensure the robustness of autonomous cyber defence solutions.