Multi-domain recommendation (MDR) aims to provide recommendations for different domains (e.g., product types) with overlapping users/items. MDR benefits from disentangling domain-shared and domain-specific user representations, but model-level disentangling methods suffer from the gradient conflict problem, which have spurred the development of new algorithms that explicitly separate global and local knowledge at both the model and embedding levels. Though effective, existing separation-based MDR methods still face three major challenges. Firstly, they are unable to capture the complete global structure information. Secondly, the learned global representation is directly applied to the recommendation of each domain, without considering its adaptability to each domain. Thirdly, the separation learning structure will lead to inconsistency between global and local representations. To address these problems, we propose UGDA, a Unified Graph-based method with Domain-specific Adaptation for MDR. Specifically, we construct a global unified graph from different domain graphs to directly learn the global representation, so as to capture more comprehensive global information. Moreover, we re-incorporate all interactions in each domain into the global unified graph, enabling the global representations to be domain-specific. Finally, we redesign the BPR loss and assign different weights to individual data sample to improve the consistency between global and local representations. Extensive experiments on representative public datasets demonstrate that UGDA consistently outperforms existing state-of-the-art MDR methods.