Enhancing Collaborative Image Classification via Spatio-Temporal Graph Neural Networks: A Proof-of-concept Study on Human Group Decisions
摘要
Collaborative decision-making is essential in expert-driven image classification tasks, where individual assessments may be inconsistent or limited. We propose a task- and label-independent spatio-temporal graph neural network (STGNN) framework to model real-time interactions among human participants during group classification. The architecture combines graph neural networks (GNNs) and recurrent units to capture relational and temporal dependencies across dynamic graph sequences, with an auxiliary contrastive loss encouraging alignment among agreeing participants, coherence with chosen options and separation from alternatives. Experiments on a collaborative web platform covered five expert classification tasks of varying complexity, including cyanobacteria and diatom identification, Ki67 scoring, HER2 grading and glomerulonephritis diagnosis. From 1,369 group classification instances by 34 participants, multiple STGNN configurations were tested, varying GNN architecture, feature initialization and temporal granularity. Stratified 5-fold cross-validation showed several configurations outperforming the majority voting (MV) baseline in global top-1 accuracy, with the best (GIN+GRU, \(T=20\) ) achieving 0.7757 vs. 0.7633 for MV. Improvements were also observed in complex tasks such as glomerulonephritis (0.4778 vs. 0.4167), HER2 (0.6100 vs. 0.5633), and Ki67 (0.8261 vs. 0.7993), demonstrating the potential of STGNNs for enhancing collaborative image classification.