We introduce a novel, scalable and energy-efficient photonic convolutional neural network (APMR-CNN) that leverages cascaded all-pass microring resonators (APMRs) to implement optical dot-product computation. Unlike conventional multi-operand microring architectures based on phase accumulation within a single resonator, APMR-CNN assigns each operand to a distinct microring, allowing modular phase-based MAC operations with reduced inter-ring interference and simplified layout routing. Each APMR neuron encodes the operand through nonlinear phase modulation and achieves signal accumulation via a unidirectional cascade, supporting passive inference without active gain or amplitude filtering. We validate the proposed architecture on the MNIST dataset, demonstrating a test accuracy of up to 98.21% with only 0.056 validation loss. The architecture demonstrates robustness under low-bit quantization (2–8 bits), maintaining over 95% accuracy even at 2-bit precision. Compared to MZI- and MORR-based photonic neural networks, APMR-CNN offers a compact layout, thermal stability, and ease of integration into wavelength-division multiplexed systems. These results establish APMR-CNN as a promising candidate for future photonic AI accelerators targeting edge inference applications.