Federated learning (FL) addresses the issue of data privacy in collaborative learning environments. Blockchain and asynchronous aggregation have been combined to address the issues of centralisation and efficiency. However, the lack of regulation of data makes FL models vulnerable to poisoning attacks. Therefore, we propose a new secure blockchain-based FL framework. The framework employs an asynchronous critical learning behaviour verification mechanism to defend against poisoning attacks. Clients construct the asynchronous learning behaviour models to illustrate its gradient variation pattern and the characteristics during local learning. Miners choose the honest client by the performance on the test dataset and remove malicious clients by comparing their critical learning behaviours with the honest client. The asynchronous weighted aggregation is employed to mitigate the impact of low-quality models during the aggregation phase. Experiments indicate that the proposed mechanism is effective against more than half of malicious attackers and outperforms existing defence algorithms, such as Foolsgold, RLR and multi-krum. Meanwhile, our proposed framework also exhibits good learning performance compared to other asynchronous federated learning frameworks, such as BAFL and FedAsync.