Crowds, Machines and Foresight: Combining Volunteer Crowds and Large Language Models to Sharpen Strategic Forecasts
摘要
This article investigates how volunteer forecasters, large language models (LLMs) and their combinations can support anticipatory foreign policy analysis in resource-constrained national security contexts. A 9-month quasi-experimental study was conducted with 34 Finnish volunteer forecasters and six LLM-based forecasters, generated by crossing model type (cloud-hosted GPT-4o versus on-premises Gemma 2) with input richness (question only, news summaries, hybrid including human forecasts). In total, 46 geopolitical outcomes were predicted. Forecasts were evaluated both for accuracy, using Brier scores, and for analytical value, through assessments by professional foreign policy analysts. The results show that volunteer forecasters and hybrid configurations achieved essentially the same high level of predictive accuracy, indicating that human input was necessary for producing the most accurate forecasts. At the same time, LLMs consistently generated rationales rated more useful by analysts, with hybrid models again performing best. These findings highlight the complementary strengths of human contextual insight and AI-generated analysis, and demonstrate practical pathways for integrating volunteer forecasters and advanced models into foreign policy workflows, particularly in small states facing rapidly expanding information demands.