Top-Down Backpropagation in Deep Feedforward Neural Networks
摘要
From a neural and cognitive standpoint, learning is known to rely on top-down information that conveys expectations about sensory events. Deep neural networks, however, have focused primarily on a bottom-up approach, where supervised learning adjusts weights based on the discrepancy between the network output and a desired target. To account for the role of both top-down and bottom-up learning, we developed a novel learning rule termed Top-Down Backpropagation (TDBP) where supervised learning is combined with a top-down component that aims to reduce the difference between the network output and expectations that are gradually adjusted during training. Compared with a standard approach, TDBP attained lower training error, better testing accuracy, and improved robustness to adversarial data. We discuss the implications of these findings for the design of artificial neural networks that account for the influence of both bottom-up sensory information and top-down expectations during learning.