The specific application effect and optimization space of pre-trained models in the field of machine translation have yet to be clarified. This study uses a combination of experimental comparison and in-depth analysis to select the GPT (Generative Pre-trained Transformer) model as a pre-trained language model and integrate it into the existing machine translation framework. By pre-training on a large-scale multilingual corpus, the GPT model can capture rich cross-language features and semantic information. Subsequently, experiments are conducted on multiple translation tasks to compare and analyze the translation output before and after the introduction of the GPT model to quantify its effect in improving translation quality. This study collects a large-scale parallel corpus covering multiple language pairs, uses the preprocessed monolingual and multilingual parallel corpora to pre-train the GPT model, builds a benchmark machine translation system based on the Transformer architecture, integrates the pre-trained GPT model into the machine translation system, runs the baseline translation system without GPT and the enhanced translation system with GPT, and collects the translation output. The experimental group that incorporates the GPT model is better than the control group in evaluation indicators such as BLEU (Bilingual Evaluation Understudy) and TER (Translation Edit Rate). The average BLEU score of the experimental group is about 98.5%, which is about 5.6% higher than the control group. In terms of TER score, the experimental group shows lower translation and editing needs. This paper provides new ideas and reference for the development of machine translation technology.