Background <p>Previous studies have found substantial heterogeneity in the effects of physical activity (PA) of different intensities on cognitive function. This study aimed to explore the association between PA levels and cognitive impairment and clarify the dose–response relationship among older adults in China.</p> Methods <p>This was a cross-sectional study, using data from the 2018 China Health and Retirement Longitudinal Study (CHARLS). We explored the association between PA levels and cognitive impairment using logistic regression, subgroup analysis, and restrictive cubic spline (RCS) analysis.</p> Results <p>Among the 3354 older participants, those who performed moderate intensity PA (MPA) showed a significant association with decreased odds ratio (OR) of cognitive impairment after adjusting for all confounding factors (OR = 0.58, 95% CI: 0.43 ~ 0.80). Subgroup analysis demonstrated consistent effects across all subgroups (<i>P</i> &gt; 0.05). The results of the RCS analysis revealed a non-linear “J”-shaped dose–response relationship between PA levels and cognitive impairment (<i>P </i>_nonlinear &lt; 0.05), with the lowest OR at around 1600 MET-min/wk.</p> Conclusion <p>MPA has greater benefits for cognitive function compared to light intensity PA, whereas vigorous intensity PA does not appear to offer additional protective benefits. PA is a dose-dependent factor associated with cognitive impairment in Chinese older adults, and approximately 1,600 MET-min/wk represents the optimal PA level for minimizing the risk of cognitive impairment. These findings can help offer more scientific PA recommendations to reduce or delay the development of cognitive impairment in older adults.</p>