Self-phase modulation nonlinearity distortion compensation in wavelength division multiplexed optical systems

This paper simulates the relative performance of various artificial intelligence (AI) techniques when applied to nonlinear distortion compensation in wavelength division multiplexing (WDM) optical communication systems. These procedures are less complex than state-of-the-art compensation methods and do not necessitate prior knowledge about the properties of data in neighboring WDM channels, which can be practically challenging. In this study, Neural Networks (NNs) were integrated into both the transmitter and receiver sections of 3- and 5-channel WDM systems, and the resulting enhancement in performance (Q-factor) was assessed across varying levels of fiber nonlinearities. While the NN stage enhances the system performance, the improvement decreases as expected with the channel number and \(\gamma\) $\gamma$ . Next, two-stage architectures that employ a transmitter side NN together with a classifier at the receiver side were modeled. For the systems examined in this paper simple decision tree structures, boosting, forests, extra trees, and multi-layer perceptron (MLP) classifiers all yielded enhanced system performance compared to simple chromatic dispersion compensation (CDC) with the only exception being Ada boosting which decreased the Q-factor for \(\gamma = 14W^{ - 1} km^{ - 1}\) $\gamma =14W^{-1}k{m}^{-1}$ . The outcomes of these investigations show that the most effective performance in highly nonlinear WDM systems is attained by employing two-stage systems, with the incorporation of random forest or extra tree AI methods at the receiver side yielding the highest results.