Network-based Intrusion Detection Systems (NIDS) are crucial in safeguarding network security, especially as cyber threats continue to evolve in complexity and scope. Despite significant advancements in IDS development, the evaluation of these systems remains inconsistent and often inadequate, particularly concerning their resilience to privacy attacks. This paper addresses this critical gap by introducing a systematic approach to assess the privacy vulnerabilities of IDS. We implement and integrate our evaluation method into the FREIDA [4, 5] tool, which is specifically designed to ensure the completeness, reliability, and reproducibility of machine learning-based IDS evaluations. To validate our approach, we conduct extensive experiments using established datasets, demonstrating the effectiveness and reliability of our evaluation methodology.