The proliferation of the Internet-of-things (IoT) has led to an unprecedented increase in versatility of wireless applications. This evolution has imposed stringent and diverse quality-of-service (QoS) requirements on modern communication systems. These requirements complicate the design of wireless systems and increase resource consumption. The problem is exacerbated by the ever-increasing scale of IoT networks. This necessitates the development of green technologies, which enable energy-efficient operations. Reconfigurable intelligent surfaces (RIS) are emerging as a promising solution to the scalability problems of next-generation systems. RISs are composed of reconfigurable elements capable of altering the communication channel with minimal power consumption. Conventional RISs have independently functioning elements and are limited in their reconfigurability. To provide advanced beamforming capabilities, beyond diagonal RISss (BD-RISs) employ fully connected elements which increase reconfiguration precision. Motivated by the scale of next-generation networks, this article formulates and solves resource allocation problems, which jointly design the transmit beamforming vectors at the base-station and the reconfiguration matrix at the BD-RIS in a multi-user multiple-input single-output (MU-MISO) system. We utilize deep reinforcement learning (DRL) to solve these problem and evaluate the impact of various objective functions on the performance of the BD-RIS aided system.