Contrast detection autofocus (CDAF) is widely adopted in portable imaging systems due to its simplicity and minimal hardware requirements. However, the axial defocus ambiguity caused by defocus blur limits the speed and precision of existing CDAF methods. To address these challenges, this paper presents a novel defocus distance ordinal regression network based on contrastive feature learning. To mitigate the effects of axial defocus ambiguity, axial position encoding is embedded and then projected into the latent feature space. Furthermore, a multi-scale attention mechanism is introduced: self-attention is applied in the encoder, and channel attention is employed in the decoder to enhance the extraction of global contextual and local detailed features, thereby optimizing the encoding-decoding process and improving defocus distance regression accuracy. Evaluated on real-world focal stack dataset, using only a single image as input, the experimental results show that the proposed method outperforms classical CDAF algorithms and mainstream learning-based autofocus methods in terms of average defocus distance error. Ablation studies further validate the effectiveness of the axial position encoding, the multi-scale attention modules, as well as other architectural components. Notably, by reformulating defocus distance estimation as an ordinal regression task, the root mean square error (RMSE) is significantly reduced from 4.02 to 2.10.