Positron Emission Tomography (PET) is an advanced nuclear medicine imaging technique widely used in the diagnosis and treatment of oncology and neurological diseases. However, PET images suffer from high noise levels due to statistical fluctuations and physical degradation factors during image acquisition. Recently, deep learning-based denoising methods have shown great performance for PET image quality enhancement. Most of these methods attempt to incorporate high-quality anatomical image (such as CT or MR), as network input to provide prior information into the PET denoising process. However, directly using CT or MR image as network input has limited effectiveness and lacks interpretability due to the significant differences between two modalities. Exploring how to make better use of anatomical prior remains a valuable research direction. In this study, we proposed an unsupervised PET image denoising framework that leverages the Bowsher prior to achieving cross-modality fusion and anatomical information extraction. Specifically, we compute the Bowsher prior using the denoised result from the Conditional Deep Image Prior (CDIP) method and the corresponding MR image. The Bowsher prior and MR image are concatenated along the channel dimension and then fed into a designed Spatial Attention Network (SA-Net) to enhance PET image quality. Experiments on both simulation and clinical datasets demonstrated that the proposed framework can effectively utilize Bowsher prior to generating high-quality PET image.