We recently introduced a scale of kernel-based greedy schemes for approximating the solutions of elliptic boundary value problems. The procedure is based on a generalized interpolation framework in reproducing kernel Hilbert spaces and was coined PDE- \(\beta \) -greedy procedure, where the parameter \(\beta \ge 0 \) is used in a greedy selection criterion and steers the degree of function adaptivity. Algebraic convergence rates have been obtained for Sobolev-space kernels and solutions of finite smoothness. We now report a result of exponential convergence rates for the case of an infinitely smooth kernel and infinitely smooth solutions. We furthermore extend the approximation scheme to the case of parametric partial differential equations by the use of position-parameter product kernels. In the surrogate modelling context, the resulting approach can be interpreted as an a priori model reduction approach, as no solution snapshots need to be precomputed. Numerical results show the efficiency of the approximation procedure for problems which occur as challenges for other parametric model order reduction procedures: non-affine geometry parametrizations, moving sources, or high-dimensional domains.