Assessment of Memory Recall and Cognitive Health Using Large Language Models

Memory impairments in everyday activities are frequently observed in older adults, particularly those experiencing early cognitive decline. In this study, we first investigate the potential of large language models (LLMs) to quantify memory recall after individuals watch videos of daily-life scenarios. Data was obtained from a prior study in which participants watched three videos (called Breakfast, Party, and Library) and were asked to recall every detail they observed. Using prompt engineering, we guided LLMs to generate recall scores and evaluated the results against human-coded ground truth scores using standard regression metrics. In addition, we examined whether the recall transcript by itself can serve as a basis for classifying cognitive health, independent other cognitive tests or neuroimaging assessments. In preliminary experimentation with GPT-3.5/Llama models and different prompts for predicting the recall score using the recall transcript, the best performance (R² = 0.53, RMSE = 12.60, MSE = 158.65, MAE = 8.46) was achieved with Llama3.3-70B on the Party event video, which has 158 actions that need to be recalled. Additionally, with two-shot prompting, the Llama3.2-11B model reached 43% accuracy on predicting cognitive health from the recall transcript of all three events. While this represents the strongest performance observed in our experiments, the restricted context window prevents the inclusion of more in-context examples. Our preliminary results indicate that, with effective prompt design and appropriate LLMs, LLMs can approximate memory recall assessments manually performed by human annotators with domain expertise, by leveraging their ability to generalize and interpret natural language. However, recall transcripts alone are not sufficient to reliably assess overall cognitive health.