Cryptanalytic Extraction of Convolutional Neural Networks

Neural network model extraction attacks pose a serious threat to the intellectual property of deep learning models. While most prior work focuses on Fully Connected Networks (FCNs), effective extraction of Convolutional Neural Networks (CNNs) remains underexplored, particularly in the hard-label setting. In this work, we propose the first systematic method for the recovery of complete CNN parameters in such conditions. By reformulating convolutional layers as sparse Block Toeplitz with Toeplitz Blocks (BTTB) matrices, we extend the model extraction attack method from FCNs to CNNs. The proposed method supports both one- and two-dimensional CNNs, handling scenarios with multiple kernels, multi-channel structures, and average pooling. To enhance computational efficiency and scalability, a kernel-centric clustering algorithm is proposed to exploit kernel parameter sharing, and a Singular Value Decomposition (SVD)-based acceleration strategy is adopted to address the computational cost of large sample sets. Moreover, we perform experiments to demonstrate that our method accurately and efficiently extracts CNN parameters, including multi-channel, multi-kernel and average-pooling layers, with a worst-case relative error of \(2^{-17.75}\) and up to \(2^{9.26}\) speedup, and recover large models LeNet-5 within practical runtime.