<p>Underwater wireless sensor networks (UWSNs) have emerged as a new paradigm of real-time organized systems, which are utilized in a diverse array of scenarios to manage the underwater environment surrounding them. One of the major challenges that these systems confront is topology control via clustering, which reduces the overload of wireless communications within a network and ensures low energy consumption and good scalability. This study aimed to present a clustering technique in which the clustering process and cluster head (CH) selection are performed based on the Markov decision process and deep reinforcement learning (DRL). DRL algorithm selects the CH by maximizing the defined reward function. Subsequently, the sensed data are collected by the CHs and then sent to the autonomous underwater vehicles. In the final phase, the consumed energy by each sensor is calculated, and its residual energy is updated. Then, the autonomous underwater vehicle performs all clustering and CH selection operations. This procedure persists until the point of cessation when the sensor’s power has been reduced to such an extent that no node can become a CH. Through analysis of the findings from this investigation and their comparison with alternative frameworks, the implementation of this method can be used to control the cluster size and the number of CHs, which ultimately augments the energy usage of nodes and prolongs the lifespan of the network. Our simulation results illustrate that the suggested methodology surpasses the conventional low-energy adaptive clustering hierarchy, the distance- and energy-constrained K-means clustering scheme, and the vector-based forward protocol and is viable for deployment in an actual operational environment.</p>