Toward a Rigorous Metric for Measuring Inconsistencies in Stakeholder Preferences in Systems Engineering

When it comes to building complex systems such as satellites, aircraft, and weapons, organizations are made up of hundreds to thousands of decision-makers who are distributed across the organizational hierarchy. These decision-makers work together to fulfill the needs of stakeholders, who can be either within or outside of the organization. In traditional systems engineering (SE) processes, stakeholder preferences are captured and communicated through textual natural language-based needs statements. While textual statements are easy to use, they have very limited abilities in terms of analysis/reasoning. Alternative approaches that are based on decision analysis (value- and utility-based methods applied to SE) facilitate capturing stakeholders’ preferences in a rigorous manner, but they are challenging to create and require a lot of time and resources without a guarantee of accuracy. Recently, researchers have proposed an approach based on formal logic, called modal preference logic (MPL), for representing and reasoning about stakeholders’ preferences in SE. Past work has also demonstrated how MPL can be used to identify inconsistencies in a set of stakeholders’ preferences. This paper builds on this past work and investigates two dimensions (inconsistency magnitude and difficulty of resolution) related to inconsistencies in a set of preferences, by presenting several examples that highlight the importance of accounting for both dimensions. Analyzing these dimensions serves as the initial step in creating a rigorous metric for quantifying inconsistency. Furthermore, this paper stresses the necessity of establishing a formal theoretical framework to guide the formulation of such a metric.