Prototype Learning Based Realistic 3D Terrain Generation from User Semantics

Customizing 3D terrain based on user semantics plays an important role in military simulation, but it is difficult to realize realistic results because of the limited ability of some simple Convolutional neural network (CNN) models. In order to meet the personalized needs of users, this article proposes a prototype learning based terrain generation network (ProTG Net). Concretely, it extracts terrain semantics based prototype features from a small number of terrain surface samples, and then transfers the pre-learned features to user customization. Specifically, a prototype learning based framework is designed, including a terrain texture generation module (TGM), prototype feature generation module (PGM), and multiple prototype features matching module (FMM). TGM is designed as the CGAN based Pix2pix (Pixel to Pixel) structure, which can generate realistic terrain textures based on user semantics, providing a reliable terrain texture data source for prototype learning. Based on the semantic terrain texture generated by TGM, multi-features are extracted in PGM including the adaptive super-pixel guided features and the terrain spatial feature. In addition, multiple feature matching strategy is proposed for achieving the better matching between prototype matching and user semantic features. Taking a public dataset of real terrain as an example, it was verified that the prototype based method can generate realistic 3D terrain and achieve user customization to obtain realistic results.