Ultra-High-Definition (UHD) image restoration presents a fundamental challenge: capturing long-range dependencies efficiently while preserving fine pixel details. Transformer-based approaches are often infeasible due to their quadratic computational complexity, and conventional spectral methods suffer from fixed basis functions and limited frequency adaptability. We introduce ChebSpec-Net, a novel spectral graph neural network architecture that addresses these challenges through adaptive processing. Our framework consists of: (1) a Group-wise Chebyshev Module (GCM) that approximates global interactions via channel-wise Chebyshev polynomial expansions, achieving linear computational complexity and reducing redundant spectral computations while maintaining spatial consistency; and (2) a Spectral Gating Unit (SGU) that applies learnable frequency gates to dynamically decompose features into degradation-specific spectral bands, mitigating band overlap and supporting detailed restoration. Extensive experiments on multiple UHD benchmarks demonstrate that ChebSpec-Net efficiently processes 4K images at 31.61 ms, achieving PSNR improvements of 0.5 to 1.2 dB over recent state-of-the-art methods.