The scene text retrieval system can search all the images containing the query text in the gallery based on the input query text and locate the position of the query text at the same time. The current state-of-the-art approach is to match query images and text instances based on visual information, which somewhat mitigates the heterogeneous differences between the two modalities and can support the zero-shot training approach. But there is no consideration of two branches of features interacting on information as a way to enhance the features. In this paper, we improve this method. First, we design a structure that enhances features using a shared-visual embedding transformer (S-VET). Text instance features and query input features are brought closer visually by a visual transformer with shared parameters and then spliced and fused with the original features to enrich the visual information of the two branch features. We then design a Multi-Scale Residual Channel Attention (MRCA) module to obtain features at different scales by using convolution kernels at multiple scales and then use channel attention to mine beneficial information from features at different scales. Facilitated by the above scheme, our method achieved a more advanced performance.