Engagement recognition aims to identify an individual’s level of participation in a particular activity, which has broad application fields, such as education, healthcare, and driving. However, the performance of engagement recognition in current methods is often compromised by excessive data and distractions. Our Behavior Capture based TRansformer (BCTR) introduces a Transformer-based video analysis approach, emphasizing frame and video level spatiotemporal details to improve engagement recognition. BCTR features dual branches for detecting static and dynamic signs of disengagements, such as eye closure and head down, through refined class tokens. This method allows the model to independently identify critical disengagement indicators, mirroring human observational techniques. As a result, BCTR not only boosts the precision but also enriches the interpretability of engagement assessments by recognizing these signs of disengagements. Extensive experimental results demonstrate that our BCTR model achieves superior performance, particularly in challenging environments rich in distractions.