Training and proficiency in job duties is an essential component of fleet readiness. Traditional classroom and on-the-job training are used to maintain these necessary skills. Although these approaches can be efficient, the effectiveness of a one-size-fits-all training approach does not provide confirmation of the individual readiness of each sailor. To address this, Adaptive Training and Assessment for Aviation Maintainers (ATAAMs) was developed that leverages an adaptive training system that transforms the learning environment to provide an individualized experience to identify skill decay, foster skill development, and offer informative after-action reviews (AAR) to trainers and trainees. A study was completed to evaluate usability and effectiveness of the ATAAMs system at addressing knowledge gaps and skill decay for specific maintenance tasks. A total sample of 111 trainee participants that were subdivided into four groups based on their job ratings. The participant’s job rating group directed what content the participant received. The outcomes of the study data indicated a statistically significant increase in initial to post remediation scores of participants that received remediation for four of the seven Technical Objectives (TOs): TO3 increase of 10.76% (n = 24, p = 0.003), TO5 increase of 10.12% (n = 90, p = 0.000), TO6 increase of 27.37% (n = 95, p = 0.000), and TO8 increase of 13.33% (n = 78, p = 0.000). This analysis found a moderate effect size (d = 0.511). The outcomes of the usability survey were also positive.