The Convolutional Neural Networks (CNNs) have proven to be highly efficient at computer vision tasks, especially image classification. However, following the success of attention mechanisms in NLP, which led to tools like ChatGPT, attention mechanisms have become essential techniques in the field of computer vision within deep learning for image classification and object detection. Attention modules aim to enhance model performance by allowing the model to focus on specific regions of images, thereby improving results. In this paper, we describe various attention methods and provide examples, ranging from self-attention used in Vision Transformers (ViT) to channel attention in Squeeze-and-Excitation Networks (SENet), and spatial and channel attention in Convolutional Block Attention Module (CBAM). We list the applications of attention-based machine learning methods applied in oceanographic image data analysis. We present a consolidated comparison of the performance of these methods based on popular performance metrics. We present a detailed survey of attention mechanisms applied in other domains for image classification tasks, highlighting their potential to address specific challenges in marine image data analysis within oceanography. We conclude that these attention mechanisms significantly enhance the capabilities of CNNs and other deep learning models in image classification tasks. This paper aims to provide a deeper understanding of various attention mechanisms in machine learning and their interrelationships, while also motivating future research in marine image analysis within oceanography to leverage these techniques for improved results.