To reduce the impact of floating garbage on river ecosystems and water resources, the YOLOv5_RFG algorithm is proposed for lightweight garbage detection in river channels. To address issues such as misdetection and omission of small targets in object detection algorithms, C2f and CBAM modules are added to the backbone network to extract critical target features. Deploying unmanned boat platforms can be challenging due to limited storage and computing resources. To solve this problem, a Ghost module is added to the neck network. The detection layer then applies PConv and removes the less efficient large object detection head to achieve model lightweighting. The experimental results show that the proposed YOLOv5_RFG model achieves the best detection performance while effectively reducing the parameter count when compared to the latest methods. This model is better at detecting floating garbage, demonstrating its capability to address the challenges posed by limited storage and computational resources on unmanned boat platforms.