Background <p>Genomic sequence-to-expression deep learning models, which are trained to predict gene expression and other molecular phenotypes across the reference genome, have recently been shown to have poor out-of-the-box performance in predicting gene expression variation across individuals based on their personal genome sequences.</p> Results <p>Here, we explore whether additional training (fine-tuning) on paired personal genome and transcriptome data improves the performance of such sequence-to-expression models. Using Enformer as a representative pre-trained model, we explore various fine-tuning strategies. Our results show that fine-tuning improves expression predictions on held-out individuals, including from held-out populations, for genes seen during fine-tuning, with comparable performance to variant-based linear models commonly used in transcriptome-wide association studies. However, fine-tuning does not improve model generalizability to held-out genes, which contain sequences and variants unseen during fine-tuning.</p> Conclusions <p>Including individual-level genetic variation and paired expression data during the training of sequence-to-expression models improves their understanding of seen variants, enabling their application to held-out individuals. However, this strategy does not improve generalizability to unseen genes, highlighting a remaining open challenge in the field.</p>