Knowledge Vault 5 /78 - CVPR 2022
Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields
Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan Barron, Pratul Srinivasan
< Resume Image >

Concept Graph & Resume using Claude 3 Opus | Chat GPT4o | Llama 3:

graph LR classDef refnerf fill:#f9d4d4, font-weight:bold, font-size:14px classDef mipnerf fill:#d4f9d4, font-weight:bold, font-size:14px classDef reflection fill:#d4d4f9, font-weight:bold, font-size:14px classDef radiance fill:#f9f9d4, font-weight:bold, font-size:14px classDef roughness fill:#f9d4f9, font-weight:bold, font-size:14px classDef normal fill:#d4f9f9, font-weight:bold, font-size:14px classDef rendering fill:#f9d4d4, font-weight:bold, font-size:14px A[Ref-NeRF: Structured View-Dependent
Appearance for Neural
Radiance Fields] --> B[Ref-NeRF improves MipNeRFs
specular reflections, view-dependent appearance. 1] A --> C[MipNeRF: semi-transparent, inconsistent
reflection approximations. 2] B --> D[Ref-NeRF: realistic specular
reflections, view synthesis. 3] C --> E[MipNeRF: glowing emitters,
foggy surface reflections. 4] A --> F[NeRF: view-dependent radiance,
challenging approximation, interpolation. 5] F --> G[Ref-NeRF: normals re-parameterize
radiance, simplify function. 6] A --> H[Ref-NeRF predicts scalar
roughness, measures points. 7] H --> I[Ref-NeRF: roughness distribution,
reflection directions, blurriness. 8] I --> J[Ref-NeRF encodes reflection
directions, spherical harmonics. 9] G --> K[Ref-NeRF needs accurate
normals, NeRF lacks. 10] K --> L[Ref-NeRF regularizer improves
normals, reduces fogginess. 11] L --> M[Ref-NeRF eliminates hidden
emitters, enhances normals, renderings. 12] D --> N[Ref-NeRF: sharp, smooth
reflections vs MipNeRFs cloudy. 13] N --> O[Ref-NeRF benefits real-time
captured scene rendering. 14] G --> P[Ref-NeRF captures geometry
through normal vectors. 15] D --> Q[Ref-NeRF enables post-training
scene appearance editing. 16] class A,B,D,N,O,Q refnerf class C,E mipnerf class F,G,J radiance class H,I roughness class K,L,M,P normal

Resume:

1.- Ref-NeRF improves upon MipNeRF by accurately representing and rendering specular reflections and view-dependent appearance.

2.- MipNeRF poorly approximates reflections as semi-transparent clouds that appear and disappear between views.

3.- Ref-NeRF produces realistic renderings of specular reflections crucial for realistic view synthesis.

4.- MipNeRF represents specular reflections as glowing emitters shining through a foggy surface.

5.- NeRF represents outgoing radiance as a function of view direction, which is difficult to approximate and interpolate.

6.- Ref-NeRF uses normal vectors to re-parameterize outgoing radiance as a function of reflection direction, simplifying the function.

7.- Ref-NeRF predicts scalar roughness at 3D locations and measures roughness at points with different roughnesses.

8.- Ref-NeRF applies a roughness distribution to reflection directions, with rougher materials having wider distributions and blurrier specularities.

9.- Ref-NeRF encodes the distribution of reflection directions as the expected value of spherical harmonics (integrated directional encoding).

10.- Ref-NeRF's reflection direction re-parameterization relies on accurate normal vectors, which NeRF lacks due to foggy geometry.

11.- Ref-NeRF applies a regularizer to produce more accurate normals and reduce fogginess by concentrating weights around the surface.

12.- Ref-NeRF eliminates hidden emitters inside objects, resulting in better normals and renderings.

13.- Ref-NeRF renders sharp reflections that move smoothly over surfaces, while MipNeRF renders cloudy specularities.

14.- Ref-NeRF shows benefits in real-time rendering of captured scenes, with sharp reflections compared to MipNeRF's cloudy reflections.

15.- Ref-NeRF accurately captures underlying geometry through extracted normal vectors.

16.- Ref-NeRF structures outgoing radiance to enable convincing post-training scene appearance editing, such as modifying roughness and color.

Knowledge Vault built byDavid Vivancos 2024