FASSET: Frame Supersampling and Extrapolation Using Implicit Neural Representations of Rendering Contents

Despite recent advances in ray tracing hardwares, ray budgets are still limited for many rendering applications, especially when global illumination is enabled. This typically results in undersampling, which manifests as low resolution and low frame-rate when displaying rendering contents. Previous works address this issue by either supersampling a low-resolution input or extrapolating new frames to increase the frame-rate. We introduce a unified and learning-based framework (dubbed FASSET) to conduct frame supersampling and extrapolation jointly, thus significantly reduce the number of pixels that are actually shaded with heavy burden. To handles two tasks simultaneously, we propose implicit neural representations for rendering contents, from which arbitrary-sized frames can be generated using latent features extracted from input low-resolution frames and some auxiliary buffers (e.g., G-buffers). By feeding them with properly warped frames, frame extrapolation with high output resolutions can be achieved as well. Since the implicit neural representations are naturally continuous and their weights are shared across all frames for a given scene, temporal coherence is largely preserved. The proposed framework allows us to only generate 1/8 pixels every two frames, thus improving the frame-rate to a maximum of \(4\times \) (Fig. 1).