RobIR: Robust Inverse Rendering for
High-Illumination Scenes

NeurIPS 2024

Ziyi Yang1     Yanzhen Chen1     Xinyu Gao1     Yazhen Yuan2    
Yu Wu2     Xiaowei Zhou1     Xiaogang Jin1‡
1Zhejiang University     2Tencent    

High Quality Albedo Estimation

High Quality Roughness Estimation

Plausible Environment Map Estimation

High-Fidelity De-Shadow

De-Shadow
Input
De-Shadow
Input
De-Shadow
Input

Multi-view Consistency Shadow Removal

Robust Performance in Real-world Scenes

Demo of RobIR

Pipeline of RobIR

HyperNeRF architecture.

Abstract

Implicit representation has opened up new possibilities for inverse rendering. However, existing implicit neural inverse rendering methods struggle to handle strongly illuminated scenes with significant shadows and slight reflections. The existence of shadows and reflections can lead to an inaccurate understanding of the scene, making precise factorization difficult. To this end, we present RobIR, an implicit inverse rendering approach that uses ACES tone mapping and regularized visibility estimation to reconstruct accurate BRDF of the object. By accurately modeling the indirect radiance field, normal, visibility, and direct light simultaneously, we are able to accurately decouple environment lighting and the object's PBR materials without imposing strict constraints on the scene. Even in high-illumination scenes with shadows and specular reflections, our method can recover high-quality albedo and roughness with no shadow interference. RobIR outperforms existing methods in both quantitative and qualitative evaluations.

BibTeX

@article{yang2023sire,
  title={Sire-ir: Inverse rendering for brdf reconstruction with shadow and illumination removal in high-illuminance scenes},
  author={Yang, Ziyi and Chen, Yanzhen and Gao, Xinyu and Yuan, Yazhen and Wu, Yu and Zhou, Xiaowei and Jin, Xiaogang},
  journal={arXiv preprint arXiv:2310.13030},
  year={2023}
}