Evaluating cognitive load using electroencephalogram (EEG) signals is a crucial research approach in the field of Brain-Computer Interfaces (BCI). However, achieving high accuracy and generalization in feature extraction and classification for cognitive load assessment remains challenging due to the low signal-to-noise ratio of EEG signals and individual differences in EEG data collection. We propose a hybrid architecture for cognitive load assessment. Our architecture first learns spatial information on a Riemannian manifold and temporal information in Euclidean space separately, then fuses the complementary information from both information streams effectively. We tested our architecture on two public datasets, and our results demonstrate the robustness of our architecture in cognitive load recognition. We achieved new state-of-the-art results on the STEW dataset with an accuracy of 96.36%, precision of 96.06%, recall of 95.56%, and F1 score of 95.79%. We reached an accuracy of 96.75%, 96.55% precision, 97.39% recall and 96.96% F1 on the EDMAT dataset, which is close to the current state-of-the-art.