This work delves into the critical realm of Face Anti-Spoofing in face recognition systems. Previous research has explored Deep Learning methodologies, encompassing End-to-End binary cross-entropy supervision, Pixel-wise auxiliary supervision, and Generative models with pixel-wise supervision. In this study, we proposed a method that synergized two main approaches: End-to-End binary cross-entropy supervision and Pixel-wise auxiliary supervision. Specifically, our method is called SCAD-Net and is made up of a Binary Classification Module and a Depth-map Decoder for pixel analysis tasks. The findings showed the effectiveness of the proposed approach, surpassing the performance of both referenced architectures and current State-of-the-art methods (SOTA). This amalgamation of methodologies not only fortifies the robustness of Face Anti-Spoofing systems but also contributes significantly to advancing the understandings and the capabilities of countering emerging spoofing techniques. Remarkably, with only 9.8 million parameters and requiring only 10 milliseconds per inference, SCAD-Net demonstrates exceptional efficiency and potential for real-world application, highlighting its suitability for deployment in resource-constrained environments.