Object classification has recently been very successful with the arrival of deep learning models. The induced models, however, require large training data, powerful computers, and are unable to detect unknown classes or incorporate new classes without retraining. Incremental learning approaches aim to incorporate new classes while preserving performance on previously learned classes, but require to know, in advance, when new classes are given to the algorithm. In this paper, we describe a continual learning approach that can learn from a single image, automatically obtain the class name, and incrementally learn and classify objects without knowing whether the testing example is from a known or unknown class. In particular, our learning agent automatically segments objects from images, consults the Internet to download relevant images and obtain the class name, builds a tree-based incremental classifier which uses support vector machines and a novel strategy based on distance between distributions to identify new classes. The proposed approach was tested against other incremental learning approaches and in real open-world conditions with promising results.