We build on the general proposal of Grzybowski et al. [16] that defines the concept of separation of two finite point sets by means of a convex set by choosing the latter, roughly speaking, as the minimum volume ellipsoid that intersects the convex combinations of all pairs of points of different class. The corresponding fitting problem is non-convex, hence we tackle it heuristically via an iterative algorithm of the block-Gauss-Seidel type solving alternatively an SDP program and a quadratically constrained (convex) program. The thusly computed separating ellipsoid is used to classify new points by means of a newly defined score based on the relative fraction of the original points that are properly separated from them. This necessarily leads to being unable to classify points—e.g, those inside the ellipsoid—making ours inherently a classifier with reject, as opposed with most proposals in the literature where a reject function is bolted upon a standard classifier. This feature can be relevant in cases where an incorrect classification may be more damaging than explicitly refusing to assign a label, indicating uncertainty. We will provide numerical experiments comparing the quality of the ellipsoidal classifier with that of standard approaches endowed with a rejection function.