Active reinforcement learning based approach for localization of target ROI (region of interest) in cervical cell images

The localization of tumour is an important factor towards the detection of malignant cervical cells. A Deep Q-Network (DQN) algorithm was implemented and RL agent was actively trained to adjust a tight bounding box around the malignant cervix nuclei. A total of 917 images from the benchmark Herlev dataset and 210 images from the Primary dataset were resized to 250 × 250 dimensions, along with their respective ground truth masked images. These images were augmented to create a dataset comprising 10,000 samples. The DQN using Active Reinforcement Learning performs using reward and punishment to encourage achievement of efficient localization. The approach overcomes the problem of Deep Learning as it ensures randomness and therefore limits the problem of overfitting. The experimental results demonstrated successful localization of cervical nuclei, giving comparable performance of state-of-the-art methods.