Recent developments in artificial intelligence (AI) have increased the demand for high-performance computational devices. However, edge devices are highly restricted in terms of computational power and memory capacity. To address this, a learning engine called swap kernel regression (SKR) [Masaharu et al. 2019] was developed that enables both inference and learning using an edge device. The key feature of SKR is ability to extend beyond the real physical capacity by collaborating with a second storage device. SKR thereby enables the virtual construction of a large-scale learning machine on a tiny device. We developed an improved version of SKR that maximizes performance by solving the thrashing problem and further reduces the computational cost. Using this improved version, we aim to achieve few-shot class-incremental learning (FSCIL), which continuously adapts to new tasks with limited samples, envisioned for on-site implementation. The proposed method uses a kernel machine that operates in a restricted environment in collaboration with a secondary storage system. The kernel parameters, which are not essential for calculating the output values for upcoming inputs, are stored in secondary storage to create a space in the main memory. The essential kernel parameters stored in secondary storage are loaded into main memory when required. Using this strategy, the system can realize recognition or regression tasks without compromising its generalization capability.