Steady-state visual evoked potential (SSVEP) has emerged as a key focus in brain–computer interface (BCI) research due to its high information transfer rate, stable signal characteristics, and non-invasive acquisition. However, existing deep learning models for decoding SSVEP often overlook the importance of frequency-specific features and typically rely on extensive subject-specific calibration, which limits their generalizability and practical applicability. To address these challenges, we propose WavNet, a novel neural network designed to recognize SSVEP signals represented in the time–frequency domain using the continuous wavelet transform (CWT). This decomposition enables simultaneous capture of both temporal and spectral characteristics, facilitating more effective feature extraction. In addition, WavNet incorporates a frequency enhancement module to emphasize informative frequency components, and a dynamic channel attention mechanism to adaptively weight spatial information across electroencephalography (EEG) channels. Experimental evaluations on a publicly available SSVEP dataset demonstrate that WavNet achieves an average accuracy of 77.49%, an information transfer rate (ITR) of 134.53 bits/min, and an F1-score of 0.771 using only 1-second EEG segments—without requiring calibration data from target users. Compared with several state-of-the-art methods, including filter bank canonical correlation analysis (FBCCA) and C-CNN, WavNet consistently delivers superior performance. These results highlight the potential of WavNet as a calibration-free, high-performance solution for real-world SSVEP-based BCI applications.