Current DNS security analysis methods primarily rely on traditional feature engineering, often neglecting the intrinsic relationships among heterogeneous DNS elements and deep semantic nuances essential for identifying advanced threats. These methods struggle to capture global higher-order relationships and typically lack targeted, interpretable explanations for complex threat patterns. To address these limitations, we propose an advanced framework integrating a Joint DNS Embedding (JDE) model with a specialized Large Language Model (LLM). The JDE model utilizes similarity-enhanced heterogeneous graph embeddings and hypergraph structures, effectively representing complex domain-IP associations to support broad Malicious Domain Detection (MDD). Complementarily, the LLM is fine-tuned specifically to analyze domain string characteristics, precisely identifying DGAs and DNS exfiltration patterns. The JDE model synthesizes spatial domain statistics with graph-derived embeddings, incorporating outputs from the fine-tuned LLM to enhance detection performance. Additionally, the specialized LLM contributes targeted explanations, significantly improving transparency and actionable insights for IP reputation evaluation (IRE). Experiments on a three-month real-world DNS traffic dataset demonstrate that our combined system achieves state-of-the-art results.