Laser processing is a rapid, versatile, and low-cost technology to print images on large surfaces. When applied to very thin films embedded with disordered metallic nanoparticles, known as quasi-random plasmonic metasurfaces, it generates colors that vary with the observation mode, making it valuable for visual security applications. Predicting these colors in different modes from the knowledge of laser processing parameters and the initial state of the metasurface can accelerate the industrialization process. However, there is no general physical model able to make this prediction accurately in various modes. In order to address this issue, this paper proposes a data-driven approach for learning deep models on experimental data able to predict the colors observed in different environments for a large range of laser processing parameters. We leverage a framework that learns jointly a shared latent space for multiple environments together with a contextual representation specific to each. This contextual representation is generated by an hypernetwork conditioned on an interpretable context vector. This context vector can be learned from few data allowing fast adaptation to new environments. This approach demonstrates that a single model can learn to predict a large range of colors across different environments. Its effectiveness is demonstrated through its ability to rapidly adapt to new scenarios with minimal data and to serve as an improved weight initializer for fine-tuning when larger datasets are available. Source code and datasets are available on Gitlab ( https://gitlab.univ-st-etienne.fr/gt101872/ECML25-Hypernetwork-ColorPrediction-metasurface .