Consequences of Missing Objectives in Applications of Multiattribute Utility Analysis

Multiattribute decision analysis allows decision makers to identify their objectives as a critical part of constructing a prescriptive model for decision making. Failure to consider all relevant objectives is one of the leading causes of poor decision outcomes (Keeney & Raiffa, Decisions with multiple objectives: Preferences and value trade-offs. Cambridge University Press, 1976). However, studies have shown that decision makers are often ill-equipped to identify objectives and may generate less than half of the objectives they later recognize as important (Bond et al., Management Science, 54(1), 56–70, 2008; Bond et al., Decision Analysis, 7(3), 238–255, 2010). The present study examines the consequences of incomplete objective sets in a broad range of multiattribute utility analysis applications from various fields such as energy planning, conservation, and disaster management. We compare agreement between models that use the originally identified objectives with models constructed from only a subset of objectives using performance metrics related to the top choice, value loss, and rank ordering of alternatives. Analyses of the MAU applications considered suggest that the consequences of missing objectives depend primarily on three factors: (1) the direction and magnitude of the relationships (correlations) between attributes defining the objectives, (2) the decision space of the alternatives considered, i.e., the competitiveness and spread of the alternatives, and (3) the steepness of the assessed scaling parameters (weights). Our analysis demonstrates in actual MAU applications that the magnitude of negative consequences associated with missing objectives is highly variable.