The integration of Inertial Navigation Systems (INS) with Global Navigation Satellite Systems (GNSS) is commonly utilized for vehicle positioning. Nevertheless, signal path obstructions can cause GNSS disruptions, undermining positioning reliability. To address this, deep learning techniques have been introduced to improve accuracy during GNSS outages, given their high precision and robust generalization abilities. This study introduces a Semi-supervised Transformer model for GNSS/INS integrated navigation, designed to operate efficiently during GNSS signal losses with minimal computational demands. This model includes two essential components: (1) A Transformer-based structure for dynamic feature extraction; (2) A self-supervised Masked Language Model (MLM) task to interpret contextual relationships and feature patterns in INS data. During GNSS outages, the model records a maximum error of 6.02 m for 40 s outages and 9.81 m for 100 s outages, notably surpassing the performance of conventional deep-learning approaches.