Today’s technological advancements have replaced virtual interviews with physical interviews and considerably changed the recruitment process. The main objective is to help recruiters choose possible candidates more efficiently by integrating modalities like emotion, head pose, and eye gaze, which analyze interviewees’ nonverbal behavior. The proposed method involves looking at the face of the interviewee during an online interview and analyzing those facial features to determine the interviewee’s emotions. The interviewee’s emotions are determined by using a lightweight CNN model. The model is evaluated on different benchmark datasets. Lightweight CNN performs better than the pre-train VGG16 model, with just a 0.32 M parameter, and achieves an accuracy of 98.05%.