Hierarchical Multi-label Learning for Musculoskeletal Phenotyping in Mice

Deep learning has shown huge potential for medical image analysis in recent years; however, the computational models require plenty of labelled image data for training. Moreover, the complex nature of the medical data makes learning very challenging and conventional models often perform subpar due to multi-label instances. This work proposes a multi-level (hierarchical) approach to detect skeletal multi-label anomalies in mice x-rays. Accordingly, a tri-level customised model was developed to train specialized layers step-by-step for targeted learning of features and improved performance at respective layers. The top (binary) level differentiates abnormal images from normal, the second level further categorizes abnormal instances into 3 sub-classes and subsequent third level performs granular analysis i.e. deep-down classification to split 3 sub-classes into 10 individual anomalies. The results shows promising performance of the proposed hierarchical approach for multi-label image data. The top level (binary) differentiates abnormal images from normal with an AUC score of 0.98. The second level differentiate three sub-classes of abnormal with mean AUC score of 0.91. For the third level, ConvNeXBase backbone shows mean AUC score of 0.81 (0.13 higher than the direct learning) whereas EfficientNetV2S backbone shows mean AUC score of 0.86 (0.11 higher than the direct learning). Hence, the overall results demonstrate the principle that level-wise hierarchical training can lead to improved performance by investigating granular details.