Machine unlearning is a discipline that seeks to make a machine learning model forget some of the information items it was trained on. This is a means to enforce the fundamental right-to-be-forgotten (RTBF), among others. However, unlearning may paradoxically increase the vulnerability of models to membership inference attacks (MIAs) on precisely the unlearned data, by creating distinguishable patterns between forgotten and non-forgotten data. This is a manifestation of the “Streisand effect”, where attempts to remove or forget information make it more noticeable. In this paper, we investigate this critical challenge and propose defense mechanisms designed to seamlessly integrate with existing machine unlearning methods. These defenses aim to obscure the distinguishability of forgotten data, thus mitigating MIA risks without compromising model performance or computational efficiency. In addition, we define rigorous evaluation criteria to assess the effectiveness of such defenses.