Adaptive compensation of nonlinear distortion in high-speed free-space optical systems

Nonlinear distortions from atmospheric turbulence and scattering critically degrade free-space optical (FSO) communication performance. We propose a hybrid Vol_LSTM framework combining the Volterra model’s nonlinear characterization with long short-term memory (LSTM) temporal dynamic modeling. This synergistic approach adaptively compensates atmospheric-induced distortions through parallel processing of instantaneous nonlinear effects and time-varying channel dynamics. Simulations and experiments demonstrate a 32 dB power budget with 31.53% performance improvement and 21% computational complexity reduction compared to conventional methods, alongside enhanced disturbance resilience. The framework’s dual mechanism of model-based Volterra filtering and data-driven LSTM adaptation provides a practical solution for atmospheric-challenged FSO systems, advancing robust optical communication design.