Social robots have been shown to benefit learning and in particular language learning. However, existing language learning robots are limited in their dialogue and adaptation capabilities: many systems use pre-scripted lessons and adaptation options. As large language models enable open-domain dialogue, a conversation practice robot now becomes feasible. Such a robot should adapt its speech to the learner’s language proficiency to make the learning more effective. For this, the robot must first accurately detect that proficiency. This study contributes to this capability by presenting a system that automatically assesses students’ speaking proficiency. Based on expert knowledge and related literature, we extract relevant features from a graded student speech dataset. We train a machine learning model on this dataset, paying particular attention to its learned weights to inform future research. We then validate the model in a human-robot interaction setting, assessing how well it generalizes from human-only training data. Our findings show that a model relying on a limited set of feature types performs sufficiently well for adaptation, with minimal degradation when applied to a human-robot interaction scenario. Future work includes further automating the proposed system and integrating it into an adaptation system, enabling a fully adaptive conversational social robot for language learning.