Stochastic and Approximate Computing for Deep Learning: A Survey

Deep learning applications are used in various technologies, such as image processing, video classification, and speech recognition. Hardware implementation of deep learning applications has gained a vast popularity for its high speed which derives from its parallel computations. Although hardware implementation of deep learning applications has higher performance and speed compared to their software counterparts, it is highly power- and area-consuming. Several methodologies have been proposed for reducing the hardware complexity and power consumption of the deep learning applications such as approximate and stochastic computing methodologies which are proposed as alternatives to exact computing methodology for fault-tolerant circuits like deep learning applications. These methodologies reduce the hardware complexity and power consumption of the circuits at a limited loss of accuracy compared to exact designs. There are several deep learning applications in the literature which are implemented by the state-of-the-art stochastic and approximate computing methodologies, although a hybrid design of these two methodologies has never been used. Using these two methodologies together in a design could give a much better result compared to using them separately, because each of them would compensate for the other one’s downsides. In this chapter, we start with the introduction of stochastic and approximate computing methodologies and their computational elements. Then, we are going to review the deep learning arithmetic units and we are going to survey some of the stochastic and approximate deep learning applications. Next, we are going to propose two area- and power-efficient hybrid stochastic-approximate designs for being used in deep learning applications, and finally, we are going to conclude the chapter and discuss the future research directions.