Eye Control and Motion with Deep Reinforcement Learning: In Virtual and Physical Environments

Attention mechanism in computer vision refers to scan, detect, and track a target object. This paper aims to develop and virtually train a machine learning model for object attention mechanism, combining object detection and mechanical automation. For this, we use Unity 3D Engine to model a simple scene in which two virtual cameras align together to realize a monocular attention in specific objects. Deep reinforcement learning, via ML-agent’s library, was used to train a model that aligns the virtual cameras. Moreover, the model was transferred to a physical camera to replicate the performance of attention mechanism.