In this paper, we propose a deviation-aware federated digital twin (DT) construction framework over wireless edge network, aiming to dynamically build high-quality DT model. In this framework, the DT to be constructed at the central server is viewed as a global DT consisting of multiple functional components, i.e., partial-DTs, created on distributed edge devices using timely collected feature data. Noticing that unpredictable physical-virtual mapping deviations in data collection and uncertainties of wireless communications can significantly degrade the system performance, resulting in relatively low model quality, increased delays, and high energy consumption. To address these issues, we formulate an online optimization problem to determine the edge device selection in federated DT construction, along with the corresponding bandwidth and transmit power allocation for maximizing the long-term DT model quality under strict delay and energy constraints. We first transform the optimization objective into minimizing the gap between the DT model quality and its theoretical optimum. Then we propose a deep reinforcement learning-based algorithm named SFDT, which dynamically solves the problem. Simulation results show the effectiveness of the proposed solution and demonstrate its superiority over counterparts.