Action recognition is a very fundamental topic in human-computer visual interaction. However, visual methods are constrained by environmental factors such as line-of-sight and lighting, which limits their application in certain scenarios. Therefore, behaviour recognition based on channel state information (CSI) has emerged, of which gesture recognition has important application value in the fields of smart home, security monitoring and virtual reality. However, existing gesture recognition systems based on Wi-Fi signals suffer from accuracy problems in cross-domain scenarios because wireless signal features are highly dependent on domain factors such as environment, location and orientation. To address this challenge, this paper proposes a new cross-domain gesture recognition model, GestureNet-BiLSTM, based on channel state information (CSI). The GestureNet-BiLSTM model achieves cross-domain recognition by extracting Body-Coordinate Velocity Profile (BVP), which is a domain-independent feature that reflects the gesture’s velocity distribution in the body coordinate system, thus avoiding the significant variations of the traditional features from one domain to another. The model first estimates the BVP from CSI signals by compressed perception techniques, and then uses deep learning networks (including convolutional neural networks and bi-directional long and short-term memory networks) to perform spatial feature extraction and temporal modelling on the BVP sequences, and ultimately achieve gesture classification. We have conducted extensive experiments in several tasks and the results show that GestureNet-BiLSTM performs well on the cross-domain gesture recognition task. The model achieves an average accuracy of 90.25% and 88.87% in tests with different positions and orientations, which significantly outperforms existing cross-domain gesture recognition methods.