Advances in robotics have broadened human-robot interaction (HRI) beyond industrial applications, enabling robots to engage directly with people in public areas. Facial attribute classification plays a crucial role in fostering more personalized and empathetic HRI. Efficient classification technology is essential in robot applications, especially those using low-cost devices such as a CPU, to ensure real-time operation. This work proposes a Superficial Convolutional Neural Network (SCNN) architecture specially developed for efficient facial attribute classification. SCNN introduces a Streamlined Feature Re-calibration Module (SFRM) to boost the feature extractor in extracting higher-quality feature maps. As a result, SCNN provides promising performance on the CelebA and LFWA datasets. Moreover, the proposed SCNN requires low computational resources and generates few parameters, which is suitable for a CPU-based device application. Integrated with a face detector, the proposed facial attribute classifier designed for a real-time scenario achieves 20.87 frames per second (FPS) on a CPU-based device with an Intel Core i7-9750H.