We present a systematic study of the family of positive definite (p.d.) kernels with the use of their associated feature maps and feature spaces. For a fixed set X, generalizing Loewner, we make precise the corresponding partially ordered set \(Pos\left( X\right) \) of all p.d. kernels on X, as well as a study of its global properties. This new analysis includes both results dealing with applications and concrete examples, including such general notions for \(Pos\left( X\right) \) as the structure of its partial order, its products, sums, and limits; as well as their Hilbert space-theoretic counterparts. For this purpose, we introduce a new duality for feature spaces, feature selections, and feature mappings. For our analysis, we further introduce a general notion of dual pairs of p.d. kernels. Three special classes of kernels are studied in detail: (a) the case when the reproducing kernel Hilbert spaces (RKHSs) may be chosen as Hilbert spaces of analytic functions, (b) when they are realized in spaces of Schwartz-distributions, and (c) arise as fractal limits. We further prove inverse theorems in which we derive results for the analysis of \(Pos\left( X\right) \) from the operator theory of specified counterpart-feature spaces. We present constructions of new p.d. kernels in two ways: (i) as limits of monotone families in \(Pos\left( X\right) \) , and (ii) as p.d. kernels which model fractal limits, i.e., are invariant with respect to certain iterated function systems (IFS)-transformations.
