Federated Learning (FL) has garnered considerable attention for its capability to train models on decentralized data while preserving client privacy. However, the heterogeneous nature of client contributions presents significant challenges in ensuring both robustness and fairness. Conventional FL approaches often fail to adequately capture these disparities, resulting in performance degradation when confronted with adversarial updates or Non-IID data distributions. In this paper, we propose the Shapley Value-based FedAdam (SVFedAdam) method from the cooperative game theory. By conceptualizing FL as a sequential cooperative game, SVFedAdam dynamically adjusts client weights based on their marginal contributions, facilitating a more robust and fair aggregation process. To mitigate the computational complexity associated with Shapley Value estimation, we propose an efficient approximation method that selectively samples a subset of clients. Moreover, SVFedAdam incorporates a hybrid aggregation strategy, where in the average-based pseudo-gradient dominates the early training stages, gradually yielding to the Shapley-based pseudo-gradient as training progresses. Extensive experiments on benchmark datasets demonstrate that SVFedAdam surpasses existing approaches in terms of accuracy, offering a practical and effective solution for adaptive FL.