Video moment localization is a challenging but essential vision-language task, requiring precise temporal segmentation and a deep understanding of multimodal textual-temporal cues. Keywords like “first” or “leaving” are crucial for distinguishing the target moment from visually similar segments. While most existing methods treat a language query as a single, indivisible entity, this chapter introduces a novel approach that decomposes the query into two distinct components: relevant cues, which aid in moment localization, and irrelevant elements, which do not contribute to identifying the desired segment. To achieve this, we propose a language-temporal attention network that dynamically learns word-level attention based on the temporal context of the video. This enables the model to selectively focus on the key parts of the query, effectively determining “what words to listen to” for accurate moment localization. By leveraging temporal context, our model improves semantic alignment between the video and the query. We validate our approach on two publicly available benchmark datasets, DiDeMo and Charades-STA. Experimental results demonstrate that our method outperforms several state-of-the-art models, showcasing its ability to handle the complexities of video moment localization tasks.