Classifier Based on Neural Networks to Determine the Patient’s Speech State

In this study, a classifier based on neural networks is proposed to assess the state of speech of patients who have undergone hemiglossectomy – surgical treatment for tongue cancer. The aim of the work is to develop a classifier based on neural networks and speech recording spectral data to assess the state of speech of patients. The classifier is required to assess changes in oral speech after surgery and after speech rehabilitation compare with speech before surgery. To achieve this goal, the following tasks were completed: development of software for converting audio recordings into mel spectrograms, creation of a classifier based on neural networks for comparing spectral data and determining the state of speech, cross-validation of the dataset and evaluation of the results of the neural network. The study discusses the use of neural networks in medicine and sound data analysis, emphasizing their importance and relevance. Results of using various architectures of the ResNet neural network were compared, including ResNet-18, ResNet-34, Res-Net-50, ResNet-101 and ResNet-152. ResNet-152 was chosen as the best model using Dropout layers with a probability of disconnecting neural connections equal to p = 0.5, and parameters epochs = 30, lr = 0.0001, as well as an image measuring 390 by 375 pixels. This model provides the highest accuracy of multiclass classification (0,67) between compared models and shows the best indicators in determining the state of speech of patients after surgical treatment and rehabilitation. http://www.tusur.ru