Collaborative and dynamic kernel discriminant analysis for large-scale problems: applications in multi-class learning and novelty detection

We present CKDA a new multi-class collaborative learning strategy based on multiple kernel discriminant analysis learners. The principle of CKDA is to share the computational and memory footprint of the kernel Gram matrix between several weak KDA learners, i.e. with a limited storage capacity. Based on bagging techniques, CKDA has the particularity to generate as many classifiers as necessary to minimize the overall empirical risk. The number of learners generated is fully data dependent and not specified beforehand, allowing CKDA to reach high compression rates. Moreover, since each KDA learner is formulated as a multi-response kernel regression problem, we derive a regularized kernel Mahalanobis distance as classification measure. Lastly, based on the definition of Mahalanobis distance, the problem of novelty detection is also addressed by computing novelty thresholds during the learning stage. Extensive experiments on several large-scale data sets show the effectiveness of the proposed algorithm both in multi-class and novelty detection problems.