Human Action Recognition (HAR) is becoming increasingly significant across various areas, including visual surveillance, human-robot interaction, healthcare monitoring, and rehabilitation. Recently, combining data from multiple sensor types has emerged as a reliable strategy to improve HAR system performance, enhancing not only accuracy but also reliability and responsiveness, in contrast to the limitations of single-modality approaches. While existing studies have extensively investigated intermediate and late fusion strategies for combining multiple data modalities, early fusion approaches have received relatively little attention. Intermediate and late fusion approaches either extract salient features from different modalities and fuse them before applying classification or perform recognition on each modality separately before combining their decisions. Although these methods have achieved remarkable results, they can still cause the loss of valuable information during feature extraction. To fully exploit the original data, this paper presents an efficient early fusion approach that directly integrates raw 3D skeletal and inertial modalities as inputs to a transformer model. Our experiments on the public UTD-MHAD dataset demonstrate superior performance, achieving \(99.4\%\) accuracy when combining skeleton with acceleration data, and \(99.6\%\) accuracy when incorporating skeleton, acceleration, and gyroscope modalities. These results validate the effectiveness of early fusion in improving HAR performance and highlight the potential of transformer-based models in this domain.