Anomaly detection is a critical component for ensuring data quality in data management; however, as continuously collected data introduces unseen normal and anomalous classes, the performance of traditional methods often deteriorates markedly. While some approaches attempt to mitigate this challenge by simulating unseen anomaly distributions, they are constrained by the quality of the generated pseudo-anomalies and fail to solve the core problem of misidentifying unseen normal instances as anomalous. We address these limitations from a novel perspective of normal invariant representation learning by proposing WAlign, which introduces a misclassification-aware weighting mechanism for the normal distribution alignment process. This mechanism mitigates the detrimental influence of misclassified instances and unlabeled anomalies on representation learning for normal instances. As a plug-and-play module, WAlign can be seamlessly integrated into two well-established anomaly detection paradigms. For each paradigm, we instantiate a lightweight base model and conduct extensive experiments on five real-world datasets. Experimental results demonstrate that integrating WAlign improves the AUC-PR by up to 3.7% and 6.1% over the respective base models, and achieves improvements reaching up to 29.7%, 35.0%, 55.9%, 79.1%, and 87.4% when further compared with 14 state-of-the-art baselines across five real-world datasets, while maintaining competitive time efficiency.