A key challenge in multi-fidelity workflows is to craft an inexpensive, yet physically meaningful, low-fidelity (LF) model that can seed subsequent high-fidelity refinement. We introduce a compact LF layer in which classical numerical schemes are re-cast as physics-based structured neural architectures: tiny trainable blocks are embedded directly within one- or two-step finite-difference formulas. Three design options are explored. First, an adaptive point is selected inside a single integration step; second, the left-hand node of the step is shifted by a neural correction; third, the trapezoidal weight is replaced by a learnable mixing coefficient that continuously spans explicit to implicit behaviour. Each adaptive element is realised by a micro-network of hyperbolic-tangent units containing fewer than ten parameters, so the entire surrogate remains lightweight and interpretable.The concept is validated on a benchmark linear oscillator driven by a wide range of parameter values. The minimal physics-based structured neural surrogates provide a robust, differentiable LF kernel that resists over-fitting under data scarcity and can be integrated seamlessly into broader multi-fidelity pipelines.