A dual-stream spatiotemporal fusion method for fish school feeding intensity identification
摘要
Accurately identifying the fish school feeding intensity is a critical step for implementing precision feeding in factory recirculating aquaculture systems (RAS). Fish exhibit a series of temporal dynamics during the feeding process, such as motion and instantaneous postural changes. However, existing single-modal visual methods for identifying fish school feeding intensity fail to efficiently capture temporal information from video, and are susceptible to severe environmental noise and water flow interference, resulting in misjudgment of feeding intensity. To this end, a dual-stream spatiotemporal fusion method (DSSFM) is proposed for fish school feeding intensity recognition. Firstly, a feature extraction network of MobileNetV3 is employed under the temporal segment network to extract features from RGB videos and optical flow map sequences. Then, the temporal connection module (TCM) is proposed to establish the inter-segment temporal correlations. Next, the cross-modality fusion module (CMFM) is proposed to achieve deep fusion of optical flow and RGB features, which introduces a selective state space model to model the inter-modality spatiotemporal consistency with linear efficiency. Within this module, the temporal enhancement bottleneck (TEB) block is proposed to focus on temporal information, including fish movement and direction information. To evaluate the proposed method, extensive experiments are conducted on the fish school feeding behavior video dataset. Experimental results demonstrate that our proposed method exceeds other mainstream single-modal and multimodal approaches, achieving a competitive performance in terms of accuracy (95.52%) and parameters (7.47 M). Therefore, the proposed method provides an effective solution for achieving high-precision, lightweight, and real-time recognition of fish feeding intensity in complex factory recirculating aquaculture environments.