Refining unified model by test-time dynamic adaptation for domain shift problem
摘要
Current domain generalization (DG) and test-time adaptation (TTA) are two primary approaches to domain shift. However, DG lacks real-time target domain adaptation capability; TTA focuses on dynamic testing-phase adjustment but is constrained by pre-trained model performance and faces method adaptation issues. Combining their strengths provides a rational solution. DG builds a robust generalization foundation for real-time adaptation by exploiting source domain information; TTA achieves target domain adaptation via dynamic adjustment, compensating for DG’s real-time limitation. Based on this, we integrate DG and TTA. First, we construct a strong generalization model using DG; then, we perform real-time adjustments to this model via TTA-based on target domain samples, while preventing catastrophic forgetting. Meanwhile, to mitigate potential computational latency induced by TTA dynamic updates, we leverage parallel computing enabled by high performance computing (HPC) to further enhance its real-time adaptation efficiency. Specifically, we propose a test-time dynamic adaptation for unified model refining method, which has two phases: the unified model acquisition phase, where we devise a feature cross-domain fusion method to construct a unified model by learning domain-generic and domain-specific features; and the test-time adaptation phase, where we design a dynamic feature extractor to set migration boundaries for catastrophic forgetting prevention, a dynamic classifier to update weights in real time from test samples, and leverage HPC low-latency computing for real-time model adaptation. Extensive experimental results on the PACS, VLCS, OfficeHome, and TerraInc datasets validate the effectiveness and superiority of our method.