Chinese Medicine Constitution Identification Based on Multimodal Deep Learning Radiomics
摘要
To address the limitations of Traditional Chinese Medicine (TCM) constitution identification, notably lack of objectivity and limited generalizability, this study proposes a novel intelligent constitution identification approach based on Multimodal Deep Learning Radiomics (MDLR), aimed at improving identification accuracy and promoting the objectification of TCM constitution identification.
MethodsData from 540 subjects were collected, including tongue images, face images, pulse, and Constitution in Chinese Medicine Questionnaire (CCMQ). Tongue and face image features were extracted using ResNet18, while the temporal features of the pulse were analyzed with a Gated Recurrent Unit (GRU). A multimodal fusion Deep Learning Radiomics (DLR) model was constructed by integrating these features and using a Support Vector Machine (SVM) for classification. Model hyperparameters were optimized via five-fold cross-validation. The performance of the model was evaluated against the CCMQ in terms of accuracy, sensitivity, specificity, and area under the ROC curve (AUC).
ResultsExperiments indicated that as the number of data modalities increased, the performance of the DLR model improved. The accuracy, sensitivity, specificity, and AUC of the MDLR model reached 0.8333, 0.8361, 0.8298, and 0.8913, respectively. Compared with the CCMQ, the MDLR model demonstrated an 8.33% improvement in accuracy.
ConclusionMDLR enables the objective quantification of inspection and palpation features in TCM. When combined with CCMQ, it facilitates more accurate constitution identification, offering a novel approach for the intelligent implementation of the TCM principle of “preventive treatment of disease”.