Improved Memristive Binarized Neural Networks Using Transformer_DCBNN Architecture with CBAM Attention Mechanism

At present network compression and hardware deployment have become a hot research topic. For network compression, binary neural networks (BNNs) use binary weights and activation function parameters instead of full precision values. However, due to the compression of bits, the process of binarization has a great impact on the accuracy and performance of the model, as well as the expression ability. To solve this problem, we propose a new network based on BNN model. In order to maintain accuracy without increasing the number of parameters, The Transformer Block and the convolutional block attention module (CBAM) module were designed to enhance the network's ability to extract and express features. It improves the accuracy by 0.3% and 8.9% on MNIST and CIFAR-10 datasets, respectively. Furthermore, a memristor array based convolution hardware implementation scheme is designed, which is helpful to realize a more efficient hardware deployment of convolutional neural networks.