Relevance assessment in search advertising is essential for enhancing user experience. However, current relevance assessment methods often perform sub-optimally in the face of redundant information in the ad titles. This issue can be largely ascribed to the problem of encoding imbalance, in which the redundant information is overvalued. To address these challenges, we propose a novel Keyphrase-Enhanced Semantic Relevance (KESR) framework. The key insight of KESR is to refine and enhance the ad title semantics by offline keyphrase extraction, thereby improving the performance of the online relevance assessment model. The KESR framework comprises two primary components: an offline keyphrase extraction module and an online keyphrase-enhanced relevance module. To achieve superior performance in keyphrase extraction, particularly for long-tail samples, we propose a LLM-based keyphrase extraction module (LLM-KPE). Furthermore, to address the discrepancy between human prior keyphrase criteria and downstream relevance task requirements for keyphrases, inspired by Reinforcement Learning from Human Feedback (RLHF), we propose a Reinforcement Learning from Downstream Feedback (RLDF) method. By aligning the LLM-generated keyphrases with those required for downstream relevance tasks, the LLM can generate keyphrases better suited for relevance tasks. Experimental results demonstrate the effectiveness of KESR in keyphrase extraction, relevance assessment, and online deployment.