FEEL is a prospective privacy-preserving framework for training a shared ML model dispersed among multiple devices. Despite its advantages, it still faces security issues, such as Byzantine attacks from malicious devices and model tampering attacks from malicious servers. To address these challenges, we present a decentralized blockchain-based FEEL (B-FEEL) architecture. Specifically, to prevent model tampering by malicious servers, we employ a secure global aggregation algorithm and a Byzantine fault tolerance consensus protocol. Implementing a B-FEEL system at the network edge involves multiple rounds of cross-validation in the blockchain consensus protocol leading to increased training latency. Therefore, we formulate a network optimization problem as a Markov decision process to minimize the latency and develop a deep reinforcement learning (DRL)-based algorithm to solve the problem. Simulation results demonstrate that B-FEEL can successfully defend against hostile attacks from both edge devices and servers, and that the DRL-based algorithm notably shortens the training latency of B-FEEL.