FPGA Implementation of Efficient Softmax Architecture for Deep Neural Networks

Neural networks have been widely used and are being improved to meet the demands of future technological advancements. Softmax is used to deliver multi-class logistic regression and classifier operations after the input data from the different Convolutional layers have been processed. Exponentiation and division operations, for example, are hardware-intensive operations in this function. The gap between highly optimized hardware-efficient neural networks and softmax implementation has been widening in recent years, resulting in a bottleneck effect. As a result, in order to work with neural networks like CNN and DNN, a hardware-efficient implementation of the function is required. For multiple values of classes, we proposed a hardware-efficient softmax architecture and implemented it in FPGAs using appropriate approximation techniques.