In recent years, technological advancements have spurred the emergence of conversational AI as a promising tool for enhancing language learning and teaching outcomes. Despite extensive exploration into the effects of chatbot technology on educational outcomes within language learning contexts, there remains a dearth of research synthesizing previous experimental findings to assess the efficacy of chatbot-supported learning in EFL education quantitatively. This study seeks to bridge this gap by examining the impact of chatbot-supported learning on various domains of English language acquisition and investigating the influence of different moderating variables on its effectiveness. A coding scheme was devised based on the theoretical framework of activity theory, and a meta-analysis was conducted, encompassing 5,031 participants from 55 experimental studies. The results indicated that chatbots moderately to substantially improved overall English language learning outcomes, with insignificant moderators, such as intervention duration, English proficiency, and educational levels. Furthermore, the analysis demonstrated that chatbots significantly enhance overall English language proficiency and most specific language skills while positively affecting affective outcomes such as learning motivation and interest. However, no significant differences were observed in the domains of English writing skills and learning anxiety. The results support the notion that conversational AI technology-mediated language instruction holds promise in revolutionizing English language learning, highlighting its positive effects in the realm of language education. This study also suggests the incorporation of a broader range of learning outcome domains in future research involving AI chatbot-based interventions in EFL education.