In recent years, active learning and PBL have been introduced in the field of education, and cooperative learning has been actively conducted. On the other hand, it has become difficult for teachers to grasp the learning status and to evaluate it appropriately. To solve these problems, this study proposes a system that can be easily introduced into educational settings and automatically estimates the activity level of each learner using video data obtained during cooperative learning. In addition to the learners’ video data, we use information on the direction of gaze and what the learners are looking at estimated from the video data, in order to take into account facial expressions and gaze information related to communication between learners. The time-series data of the learners’ video data and gaze information are input to the deep learning model, and the activity level of each learner is estimated. We conducted an evaluation experiment using video data collected from a subject experiment during cooperative learning and showed the usefulness of the proposed method.