The proliferation of offensive language on social media poses significant challenges to social harmony. As a result, offensive language detection is crucial for maintaining a civilized online environment. Current research mainly focuses on identifying explicit offensive content in isolated statements. However, the explainability of implicit offensive language in dialogues remains underexplored. To this end, we propose a new task: Dialogue Implicit Offensive Language Explanation (DIOLE). To support this task, a novel Implicit Offensive Dialogue Explanation Dataset (IODED) is introduced, which is generated using ChatGPT and validated by humans. Besides, an advanced baseline model, the Dual-path Attention and Knowledge-Injected Explanation Model (DAKIEM) is designed to provide plausible explanations for implicit offensive language in dialogues. It learns contextual representations and incorporates external knowledge to enhance understanding of implicit offensive content. Extensive experiments demonstrate the superior performance of DAKIEM, highlighting its potential as a powerful baseline for the DIOLE. In addition, IODED contains a comparable number of non-offensive instances, and the experimental results indicate that IODED is a valuable resource for detecting offensive language in dialogue.