A Hybrid-Multimodal Mental Health Chatbot for Psychological Counselling
摘要
It is very difficult to assess the mental state of a person without considering the factors such as his facial emotions, body language as well as intonation. This is more so important when we try to use an interactive way to engage and help mentally disturbed patients through chatbots. Researchers have built chatbots that considers two of the three features mentioned above, which showed some early signs of results, but was too general for the unique set of emotions a human face could show. In this paper a framework for counselling using triple characteristics (verbal communication, facial emotion recognition and body pose emotion intensity recognition) is proposed. It uses multiple machine learning algorithms and an 18-point intensity scaling to understand the current state a patient is in providing a consultation from a hybrid dataset, with a response vetted by psychologists. The responses have been found to provide highly intricate answers tailored to the exact requirements of patients, and show promising results.