This chapter examines the ethical, legal, and societal implications (ELSI) research that was part of the United States Defense Advanced Research Projects Agency (DARPA) "In The Moment" (ITM) program. Initiated in 2022, ITM is a multi-phase basic research program with the objective of advancing research in algorithmic decision-making (ADM) systems for high-stakes domains where there is no agreed-upon right answer. This chapter shows why ELSI considerations are essential for building trust and adoption of autonomous systems, and how these concerns extend beyond technical capabilities to fundamental questions about human judgment, accountability, and the nature of delegation in life-or-death scenarios. Drawing on findings from ITM expert convenings, we present insights into the challenges of operationalizing human-AI collaboration in high-stakes domains where there is no agreed-upon "right answer." The chapter concludes by identifying emerging challenges and outlining future research directions for developing autonomous systems that can be meaningfully aligned with human operators' intentions, values, and judgment.