Federated Learning (FL) has attracted considerable attention due to its ability to train machine learning models collaboratively without sharing raw data. However, the computational limitations of devices involved in FL often lead to slow performance and delays in the training process. To address these challenges, we propose a novel approach leveraging a Deep Reinforcement Learning (DRL) technique to adjust the training intensities of participating edge devices (EDs) dynamically. Additionally, we integrate Bayesian Optimization (BO) techniques to optimize the allocation of local training iterations based on the computational capabilities of each client device. The proposed approach enhances FL performance by effectively managing the training workload across distributed edge devices, thereby improving the convergence of the global model. Through extensive simulations, we demonstrate that the proposed approach mitigates the computational constraints of individual devices and facilitates faster model convergence in FL scenarios.