Accurately determining fish populations is crucial for marine science, aiding in species tracking, fish behaviour studies, and sustainable aquaculture practices. This study evaluates the effectiveness of automated fish detection using You Only Look Once (YOLO) models, particularly YOLOv8 and YOLO-NAS, using benchmark datasets (DeepFish and OzFish) and two novel datasets from the South African Institute for Aquatic Biodiversity (SAIAB). The results indicate that YOLOv8 models consistently outperform YOLO-NAS models, effectively addressing challenges like poor image quality and varying environmental conditions. This study concludes that YOLOv8 models are better suited for underwater fish detection, offering robust performance and highlighting the potential of advanced deep learning models in enhancing ecological studies and sustainable fishery management.