Graph contrastive learning (GCL) methods are dedicated to modeling the invariant information from graphs via well-crafted graph augmentation or stochastic encoder perturbation. Although prevailing methods have achieved great progress, we argue that they overlook the essential topological invariant information, referred to as topological rationale. In this regard, we conduct exploratory experiments that visually demonstrate the deficiency of GCL methods in capturing topological rationale and reveal the positive correlation between this deficiency and the discriminability degeneration of GCL methods. To this end, we introduce a novel plug-and-play approach, termed Topological Rationale-enhanced Graph Contrastive Learning (TRGCL). Specifically, TRGCL integrates the node-level and substructure-level topological rationale learning modules in the topological rationale learning stage, thereby empowering the GCL encoder to capture topological invariant information sufficiently. Furthermore, we introduce a semantic-orthogonal adaptive weighting module to ensure that the derived topological rationale remains complementary to semantic information. Theoretically, we revisit the paradigm of GCL from the causal perspective and substantiate the theoretical validity of TRGCL. Experimental results on various datasets in the domains of social networks and biochemical molecules demonstrate the effectiveness of TRGCL.