The performance and fairness of Federated Learning systems depend on the client selection strategy, a challenge for Network Intrusion Detection Systems in heterogeneous Internet of Things/Edge environments where client data is often Non-Independent and Identically Distributed and device capabilities vary. This work introduces RAMPART-FL, a system where a Reinforcement Learning agent dynamically chooses clients to optimize competing objectives. The agent is guided by a multi-criteria reward function designed to maximize model performance, promote participant fairness by penalizing client starvation, and ensure resource efficiency by penalizing the selection of costly devices. We perform a comparative analysis of our multi-criteria RL agent against strategies that optimize for single objectives like performance, fairness, or resources. Using a Generative Adversarial Network-based Network Intrusion Detection System trained on the ToN-IoT dataset, our results demonstrate that while all strategies converge to a similar peak F1-Score, the multi-criteria agent does so with greater efficiency. It effectively learns to balance model quality, participant fairness, and resource costs, achieving an F1-score of approximately 0.82, comparable to a centralized baseline. We show that the agent progresses through distinct learning phases, from initial exploration to stable, long-term management, demonstrating its adaptability in dynamic FL environments.