Lifelong learning with selective attention over seen classes and memorized instances

Catastrophic forgetting challenges lifelong classification learning of modern neural networks, especially when observations arrive from a data stream and the boundaries of classification tasks are unknown. In this study, we focus on the online task-free setting and formulate the continual learning of a sequence of classification tasks as a dynamically weighted loss minimization problem. Specifically, we present Learning with Selective Attention over seen Classes, which minimizes the empirical loss on each seen class, and enforces the losses of different classes to be reconciled with gradient-weighted attention. To ensure an efficient and accurate loss estimation without reserving and rehearsing all the previously seen data, we further propose Learning with Selective Attention over memorized Instances, which relies on a hard attention to select replay samples sharing both stream-sensitivity and distribution-diversity from an elaborately maintained exemplar reservoir. Experimental results on four lifelong learning benchmarks validate the superiority of the proposed approach.