Combined Global and Local Information Diffusion of Neural Processes

Neural Processes (NPs) are a novel technique that combines neural networks and stochastic processes. NPs map the distribution function of target sample points from context sample points in input-output pairs. During the decoding process, context sample points are used as conditional inputs to generate distributional outputs for target sample points. Prior research has focused on issues within the NPs model structure, such as using only the average of context sample points or a single latent variable in the encoding module. However, this paper places emphasis on the data of context sample points within the NPs model and introduces a new model called Diffusion Neural Processes (DNPs). This model employs a controlled gradient sampling approach to ensure the quality of locally informative context sample point sampling. Additionally, the model introduces a pre-training noise method to capture global information from context sample points, thereby enhancing the complexity of their latent distribution representation. DNPs demonstrates excellent performance in 1D and 2D data, showcasing the model’s potential across a range of applications.