Designing an Automated Machine Learning Approach for Transformer Architecture in Education and Non-STEM Research Settings

With the recent rapid development of transformer architecture, Artificial Intelligence has become a household name. However, whilst this has seen paralleled growth in STEM research fields, traditional obstacles like programming skill sets and expert understanding of model architectures have prevented any considerable movement into non-STEM research and education settings. As a result, the full potential of transformer architecture and attention mechanisms are often negated by the perceived difficulty of the field. In this Education System Design Paper, we submit that Automated Machine Learning principles can provide a solution to this problem. We propose a novel education framework design named Edu-AutoML, that utilises the benefits of automation, enabling users to learn using a system that adapts based on their past experience. Being designed for research and education, our prototype design demonstrates how an offline web-based Python platform could provide easier model training and distribution in a classroom setting by exploiting the benefits of automated hyperparameter tuning and data pre-processing techniques.