Fast and Accurate Pupil Localization in Natural Scenes

The interferences, such as the background, eyebrows, eyelashes, eyeglass frames, illumination variations, and specular lens reflection pose challenges for pupil localization in natural scenes. In this paper, we propose a novel method comprising improved YOLOv8 and Illumination Adaptive Algorithm (IAA), for fast and accurate pupil localization in natural scenes. We introduced deformable convolution into the backbone of YOLOv8 to enable the model to extract the eye regions more accurately, thus avoiding the interference of background outside the eye on subsequent pupil localization. The IAA can reduce the interference of illumination variations and lens reflection by adjusting automatically the grayscale of the image according to the exposure. Experimental results verified that the improved YOLOv8 exhibited an eye detection accuracy (IOU \(\ge\) $\ge$ 0.5) of 90.2%, while the IAA leads to a 9.15% improvement on 5-pixels error ratio \({\varvec{e}}_{5}\) ${\mathit{e}}_5$ with processing times in the tens of microseconds on GPU. Experimental results on the benchmark database CelebA show that the proposed method for pupil localization achieves an accuracy of 83.05% on \({\varvec{e}}_{5}\) ${\mathit{e}}_5$ and achieves real-time performance of 210 FPS on GPU, outperforming other advanced methods.