This article considers the problem of classifying data directly via simulation, when the underlying model is too difficult to express as a neat analytic expression. We begin by considering how machine learning is currently used to classify data, and what is wrong (if we call it that) with such an approach. After prescribing the antidote, we will consider techniques for classifying data directly from simulations. In this article we will only consider the “classical” technique of Karhunen-Loeve transforms as well as a more modern machine-learning approach.