Identifying and tracking mobile objects in an underwater environment is a challenging task. Traditional methods cannot differentiate the object from the background due to the losses induced by inherent properties of light. In this regard, researchers across the globe developed several deep learning models that adhered to convolutional kernels and their modified forms to tackle the same. However, such kernels incur losses due to uncertain, fuzzy, and poorly defined boundaries of objects inside the water. Recent advancements in graph learning have eliminated the loss typically associated with convolutional kernels, creating new opportunities for moving object identification. This study presents an end-to-end moving object detection architecture to analyze intricate underwater scenes. We adhered to a ResNet-50 backbone in the proposed architecture to project the video frame to feature space. Graph learning is used to retain the structural information of the object by projecting from feature space to graph space. Multiple aggregators facilitate the seamless transfer of information among neighbouring nodes, alleviating noise induced by deep architectures. The refactored latent vector is transformed to image space to detect the moving object(s) from the given scene. The proposed method is evaluated against twenty-four state-of-the-art algorithms on the benchmark datasets, outperforming all existing methods.