In this paper, we aim to explore efficient methods for conducting privacy-preserving neural network inference by adopting the multi-party computation technique paradigm. Currently a main efficiency bottleneck of this paradigm is due to heavy overhead of nonlinear functions, which refers to the comparison functions, ReLU and Maxpool, in the scenario of secure neural network inference. As shown in Falcon (PoPETs’21), ReLU and Maxpool can be reduced to the primitive of the Check Zero problem i.e., checking whether there are zeros in a set of data. Thus a feasible way to enhance the performance of ReLU, Maxpool as well as the whole privacy-preserving neural network inference is to design a more efficient secure protocol for the Check Zero problem. Our main result in this paper is that we present two efficient 3-party Check Zero protocols, one for semi-honest security and one for malicious security of honest majority. Based on our efficient Check Zero protocols, we achieve more efficient ReLU and Maxpool. Due to our Check-Zero protocols, the number of communication rounds in ReLU is reduced from \(\log l+5\) (where l is the bit length of single data) to 5 rounds in the semi-honest setting and to 6 in the malicious setting. Compared to the state-of-the-art, Falcon, we improve the online runtime of ReLU by about \(1.5-1.8\) \( \times \) , and the runtime of Maxpool by approximately \(1.6-1.7 \) \(\times \) . We further validate the improvement through experiments involving privacy-preserving neural network inference using our nonlinear primitives. Compared to Falcon, we improve online runtime by about \(1.4\) \( \times \) .