TriNeRFLet: A Wavelet Based Triplane NeRF Representation
Multiscale 2D wavelet representation with high quality reconstruction and super-resolution
ECCV 2024
Tel Aviv University
Overview
In this work, we propose TriNeRFLet, a 2D wavelet-based multiscale triplane representation for NeRF, which closes the 3D recovery performance gap and is competitive with current state-of-the-art methods. Building upon the triplane framework, we also propose a novel super-resolution (SR) technique that combines a diffusion model with TriNeRFLet for improving NeRF resolution.
Example of the multiscale property of the wavelet representation. On the right, we the same feautre plane at different resolutions. On the left we see the wavelet representation of the plane at the left, which is of the highest resolution. Note that its wavelet representation include the wavelet representations of the lower-resolution planes (the colors corresponds to the matching between each plane and its wavelet representation).
TriNeRFLet training scheme. First, wavelet features are transformed into the Triplane domain. Next, pixels are rendered using these features in order to fit them to their ground-truth values. The high frequencies channels of LH, HL and HH from all wavelet levels get regularized by L_1 loss.
SD-refine process . First, a HR image is rendered. To improve its quality, noise is added to it and then it is plugged into Stable Diffusion upscaler with its LR version for conditioning. The result, which is a refined HR image is added to the HR-set.
TriNeRFLet super-resolution. Low-resolution TriNeRFLet renders LR images using the LR version of the wavelet features, and fits them to the given low-resolution images. High-resolution TriNeRFLet renders HR images using all wavelet levels, and fits them to HR images from HR-set.
Display Data Quality
3D Reconstruction Results
Blender Super-Resolution Results
LLFF Super-Resolution Results
Citation
If you use this work or find it helpful, please consider citing: (bibtex)
@inproceedings{khatib2025trinerflet,
title={TriNeRFLet: A Wavelet Based Triplane NeRF Representation},
author={Khatib, Rajaei and Giryes, Raja},
booktitle={European Conference on Computer Vision},
pages={358--374},
year={2025},
organization={Springer}
}