The location of unmanned aerial vehicles (UAVs) in GPS-denied environments remains a central challenge in cross-view geolocation tasks, primarily due to large domain discrepancies between aerial and ground-level imagery and the presence of visually similar but semantically distinct scenes. Inspired by multimodal machine learning, we propose a single-stream pyramid transformer network called MixFP. The proposed architecture initially employs convolutional layers to enhance low-level feature representation, followed by a cross-attention mechanism that enables effective interaction and refinement of features across modalities while suppressing irrelevant noise. To further augment the discriminative capacity of the model, we integrate FcaNet to process low-resolution feature maps at multiple stages of the backbone, and employ a feature pyramid network (FPN) to facilitate the fusion of multi-scale representations. Additionally, we incorporate DySample into the pyramid structure to enhance sampling flexibility and boost overall performance. Experimental results on the UL14 benchmark dataset demonstrate the superiority of MixFP, yielding an improvement in the RDS metric from 76.25 to 81.82. Furthermore, MixFP exhibits significant gains in meter-level accuracy (MA), with 3m accuracy increasing by 30.43%, 5m accuracy improving by 18.84%, and 10m accuracy rising by 9.9%.