Adaptive Planning: Comparing Human and AI Responses in Premortem Planning

Premortems are a structured analytic technique designed to evaluate a plan before it is implemented. They have been used to improve estimates and generate alternative predicted outcomes. One psychological mechanism that underlies the Premortem is prospective hindsight with people generating reasons for a plan’s failure. The reasons reduce people’s confidence and overconfidence in their plans relative to other plan critiquing methods. In this case study, we qualitatively compare student-generated responses for evaluating a plan and a large language model-generated (e.g., Chat GPT-4) evaluation of the same plan. To the extent that the reasons for failure generated by the large language model are similar or different to the human participants is the first step in a conceptual evaluation. The results have implications for a variety of structured analytic techniques to improve human-AI teaming in decision making. Furthermore, they provide initial suggestions for where adaptive systems could support collaborative intelligence.