One of the main challenges in data stream learning are concept drifts, which encompasses changes in both the data distribution and the posterior class distribution over time. Several approaches based on online neural networks have been proposed for Online Supervised Classification (OSC) in data stream learning. However, these techniques do not adapt the optimization algorithm to incoming concept drifts, which can lead to weight updates with misleading or obsolete information. In fact, several momentum-based algorithms, such as AdamW, employ past gradient information in the form of exponential moving averages and depend on several user-defined constant hyperparameters for controlling these averages. However, these gradients and hyperparameters may not be meaningful for new concepts, which causes the optimizer to become obsolete. Moreover, the tuning of these hyperparameters may require the availability of the entire dataset, which is not possible in data stream learning. In this paper, we extend the AdamW optimizer and propose the Online Adaptive Moment Estimation (OAdam), an algorithm for training online neural networks in data stream learning that automatically adapts to concept drift without hyperparameters. It employs bias and variance to detect how well the model is representing and learning the data and evolve its training accordingly. When a concept drift occurs, it is capable of adapting the network’s momentum, regularization and step size to better fit the incoming changes in class distribution. Our experiments confirmed that our method is able to react and adapt to concept drifts more effectively than other approaches, producing better predictive performance.