With the rapid advancement of AI technology, machine learning applications have permeated numerous domains. Consequently, machine unlearning has emerged as a crucial research domain driven by user concerns over privacy and security, alongside the increasing threats to machine learning systems. GDPR and CCPA regulations have empowered users to exercise their right to data erasure, thus catalyzing further advances in machine unlearning. This paradigm aims to facilitate the removal of specific data from trained models, which may contain sensitive or harmful user information. Despite advances made in previous studies, there remains a significant potential for improvements in both flexibility and effectiveness. In this study, we design machine unlearning algorithms from a knowledge perspective, considering both the retain set and the forget set. We propose a knowledge-based unlearning framework called KaRe that aligns the model’s performance on the forget set with that on unseen data. It also uses repair to ensure the model maintains similar performance on the retain set as a retrained model. KaRe flexibly supports various types of data accessibility and data granularity requirements for unlearning tasks. We present experimental findings in conjunction with a comparative analysis with the baseline results. Our experimental results demonstrate the flexibility and effectiveness of our framework in performing machine learning tasks across various deep networks and diverse domains.