Let G be a connected graph. The toughness of G is defined as \(t(G)=\min \left\{ \frac{|S|}{c(G-S)}\right\} \) , in which the minimum is taken over all proper subsets \(S\subset V(G)\) such that \(c(G-S)\ge 2\) where \(c(G-S)\) denotes the number of components of \(G-S\) . Confirming a conjecture of Brouwer, Gu (SIAM J Discrete Math 35:948–952, 2021) proved a tight lower bound on toughness of regular graphs in terms of the second largest absolute eigenvalue. Fan, Lin and Lu (Eur J Combin 110:103701, 2023) then studied the toughness of simple graphs from the spectral radius perspective. While the toughness is an important concept in graph theory, it is also very interesting to study |S| for which \(c(G-S)\ge l\) for a given integer \(l\ge 2\) . This leads to the concept of the l-connectivity, which is defined to be the minimum number of vertices of G whose removal produces a disconnected graph with at least l components or a graph with fewer than l vertices. Gu (Eur J Combin 92:103255, 2021) discovered a lower bound on the l-connectivity of regular graphs via the second largest absolute eigenvalue. As a counterpart, we discover the connection between the l-connectivity of simple graphs and the spectral radius. We also study similar problems for digraphs and an edge version.