Graph Transformers have shown strong potential in modeling complex dependencies on graph-structured data with fully-connected attention. However, they suffer from quadratic computational complexity to graph nodes and are limited in handling large-scale graphs. Recent works usually consider linear or sparse attention mechanisms but could disrupt graph structures and miss important long-range interactions. To address the problem, in this paper, we propose a novel implicit graph transformer that recursively leverages a single layer of subgraph-based multi-granularity sparse attention to simultaneously achieve computation, parameter, and memory efficiency. Specifically, we resort to deep equilibrium modeling based on the single-layer design to achieve fixed-point graph representation for computation, parameter, and memory efficiency. To further preserve structural information and capture long-range dependencies, we make dynamic subgraph partition to enable sparse intra-subgraph attention for local aggregation and inter-subgraph attention guided by importance-based sampling for global information exchange. The proposed method significantly reduces memory and parameter costs, and allows a linear computational complexity with the number of nodes. Extensive experiments demonstrate that the proposed method consistently outperforms existing Transformer-based graph models in both accuracy and efficiency in node classification.