Action anticipation aims to forecast upcoming human actions from previously observed video frames. Conventional encoder-decoder approaches dominate this task, yet they pass raw encoder output frame-level features directly to the decoder, overlooking the high predictive uncertainty of visually ambiguous frames, which often degrades prediction quality. Inspired by uncertainty modelling in probabilistic learning, we present a plug-and-play Adaptive Uncertainty Masking Learning framework (AUM-Net) that can be inserted between any encoder and decoder to down-weight ambiguous frame contributions and dynamically adjust gradient of each frame during training. The proposed framework consists of three lightweight parts. Stochastic Feature Generator (SFG) replaces the last encoder layer with Gaussian probabilistic layers, producing multiple feature samples for each input frame. Uncertainty Masking Gate (UMG) leverages samples obtained from SFG to compute per-frame uncertainty and perform masking operation. An adaptive gradient enhancement module amplifies the contribution of high-uncertainty frames by applying an Uncertainty-Based Difficulty-Boost Loss we designed. In this way, AUM-Net supplies the decoder with more reliable features while ensuring that high-uncertainty frames receive stronger supervision during training. Extensive experiments on EK100 and THUMOS’14 demonstrate the effectiveness and broad generalization capability of our AUM-Net framework.