Network connectivity exposes infrastructure and assets to vulnerabilities that attackers can exploit. Protecting network infrastructures against cyberattacks requires stringent security preventions and measures. Consequently, this incur costs in terms of money, time, and energy. In order to address this challenge, we formulate a joint Security-vs-QoS optimization problem for Intrusion detection systems (IDS) in 5G networks. The problem considers both security and QoS factors when dynamically selecting effective solutions to address detected attacks. In this work, formulation of the optimization problem takes into account both joint security and QoS utility function based on Cobb-Douglas theory. Consequently, a series of parameter analysis is performed to investigate the effect of varying relevant parameter values ( \(\alpha _1\) and \(\alpha _2\) ) on the cost function. Furthermore, a mathematical proof for the value of \(\alpha _1\) for optimality is also included. Before conclusion of this work, we present a small case study with feasible solution using the Binary Integer Programming method and the prominent Quadratic Unconstrained Binary Optimization (QUBO) strategy.