Optimized libraries for domains such as Fast Fourier Transforms (FFT) or deep learning provide specialized kernels to unlock the full performance potential of accelerators. However, specializing kernels at compile time and including many different variants to provide performance over the diverse set of accelerator hardware in today’s accelerated computing ecosystem leads to prohibitively large library binary sizes. In this work, we propose to partially postpone kernel specialization from compile time to runtime to address this issue while still providing the same adaptability to different devices. By specializing the kernels at runtime, we aim to improve performance by generating more specialized code while significantly reducing library size. We focus on the SYCL programming model and develop a SYCL extension for dynamic private memory allocation leveraging SYCL specialization constants. We evaluate the impact of our approach by applying it to two SYCL research libraries, namely portFFT, an FFT library written in SYCL, and portDNN, a SYCL library providing kernels for deep neural network operators. In the case of portFFT, we see a 1.25x speedup in one of the benchmarks updated to use our extension. For portDNN, generating kernels at runtime instead of at compile time helped reduce library size by a factor of 1.46x.