Wi-Fi channel state information has gained traction for human activity recognition, localization, and physiology monitoring, due to the positive results found. However, most of these solutions use different sampling rates, input durations, and neural networks, making the data scalability and robustness of future adaptation challenging. To that extent, this paper explores using adaptive pooling layers, namely spatial pyramid pooling, to reduce additional weights and training to handle varying packet arrival rates and activity durations. On a self-collected dataset with 20 participants, it is shown that spatial pyramid pooling achieves accurate human activity recognition with changing sampling durations ranging from 0.1 to 10 and packet arrival rates of 1 to 100 Hz, with an \(F_1\) -score \(> 0.80\) in certain scenarios. These observations are validated on three different datasets for human activity recognition and sign language gestures with different collected transmission rates. The evaluation shows a trade-off in accuracy versus scalability for different packet arrival rates and frame durations, along with a discussion on the possibilities of quickly retraining when changes occur in the context of joint communication and sensing in Wi-Fi channel state information systems.