Knowledge distillation is the process of transferring knowledge from a teacher network to a student network. However, previous methods have neglected the robustness issue of the student model when it is trained solely on clean data samples. We introduce a hierarchical integration framework that enhances the robustness of the student model. Our validation indicates that the decoupling of low-level semantic knowledge from high-level semantic knowledge within this framework is key to improving robustness. In the distillation process, to more effectively extract feature and logit knowledge with a focus on different levels of semantic information, we have designed a feature knowledge extraction mechanism based on initial feature fusion and a logit knowledge extraction method using adaptive temperature normalization. These two knowledge extraction methods provide key knowledge with distinct focuses for subsequent decoupling. By applying our method to classification tasks, we have achieved a marked improvement in the robustness of the student network.