A Theory of Fermat Paths for Non-Line-of-Sight Shape Reconstruction

Shumian Xin, Sotiris Nousias, Kyros Kutulakos, Aswin Sankaranarayanan, Srinivasa G. Narasimhan and Ioannis Gkioulekas.

**Concept Graph & Resume using Claude 3 Opus | Chat GPT4o | Llama 3:**

graph LR
classDef fermat fill:#f9d4d4, font-weight:bold, font-size:14px
classDef nlos fill:#d4f9d4, font-weight:bold, font-size:14px
classDef transients fill:#d4d4f9, font-weight:bold, font-size:14px
classDef reconstruction fill:#f9f9d4, font-weight:bold, font-size:14px
A[A Theory of

Fermat Paths for

Non-Line-of-Sight Shape Reconstruction] --> B[Fermat paths: shortest, longest

light paths discontinuities. 1] A --> C[Non-line-of-sight imaging: reconstructing

hidden objects. 2] C --> D[Virtual source, sensor:

wall/diffuser scatters light. 3] A --> E[Transient imaging: measuring

light intensity over time. 4] E --> F[Discontinuities in transients:

Fermat path lengths. 5] B --> G[Specular, boundary Fermat paths. 6] E --> H[Transients: Fermat, non-Fermat photons. 7] B --> I[Identifying Fermat lengths

from transient discontinuities. 8] B --> J[Fermat paths: geometry-dependent,

BRDF-independent. 9] C --> K[Sphere constraint: object point,

virtual source, path length. 10] K --> L[Tangent sphere for

specular paths. 11] B --> M[Fermat flow: path gradient,

source-surface direction. 12] M --> N[Reconstructing point: intersect

sphere, gradient line. 13] B --> O[Specular paths give

surface normal. 14] M --> P[Estimating Fermat gradients

from nearby lengths. 15] C --> Q[Reconstruction: scan, measure,

detect, estimate, reconstruct. 16] Q --> R[Surface from point cloud

via Poisson reconstruction. 17] C --> S[Applicable to various

transient imaging systems. 18] C --> T[Reconstructing diverse objects

around corner high accuracy. 19] C --> U[Reconstructing coin around corner,

through diffuser fine detail. 20] class A,B,G,I,J,M,O,P fermat class C,D,K,L,Q,R,S,T,U nlos class E,F,H transients class N reconstruction

Fermat Paths for

Non-Line-of-Sight Shape Reconstruction] --> B[Fermat paths: shortest, longest

light paths discontinuities. 1] A --> C[Non-line-of-sight imaging: reconstructing

hidden objects. 2] C --> D[Virtual source, sensor:

wall/diffuser scatters light. 3] A --> E[Transient imaging: measuring

light intensity over time. 4] E --> F[Discontinuities in transients:

Fermat path lengths. 5] B --> G[Specular, boundary Fermat paths. 6] E --> H[Transients: Fermat, non-Fermat photons. 7] B --> I[Identifying Fermat lengths

from transient discontinuities. 8] B --> J[Fermat paths: geometry-dependent,

BRDF-independent. 9] C --> K[Sphere constraint: object point,

virtual source, path length. 10] K --> L[Tangent sphere for

specular paths. 11] B --> M[Fermat flow: path gradient,

source-surface direction. 12] M --> N[Reconstructing point: intersect

sphere, gradient line. 13] B --> O[Specular paths give

surface normal. 14] M --> P[Estimating Fermat gradients

from nearby lengths. 15] C --> Q[Reconstruction: scan, measure,

detect, estimate, reconstruct. 16] Q --> R[Surface from point cloud

via Poisson reconstruction. 17] C --> S[Applicable to various

transient imaging systems. 18] C --> T[Reconstructing diverse objects

around corner high accuracy. 19] C --> U[Reconstructing coin around corner,

through diffuser fine detail. 20] class A,B,G,I,J,M,O,P fermat class C,D,K,L,Q,R,S,T,U nlos class E,F,H transients class N reconstruction

**Resume: **

**1.-** Fermat paths: Locally shortest or longest paths light travels in non-line-of-sight imaging, producing discontinuities in measured transients.

**2.-** Non-line-of-sight imaging: Reconstructing object shape outside sensor's line-of-sight by analyzing light transport, either around a corner or through a diffuser.

**3.-** Virtual source and sensor: Wall/diffuser points scattering light to/from the non-line-of-sight scene, allowing indirect imaging.

**4.-** Transient imaging: Measuring histogram of light intensities over time (transient) instead of a single intensity.

**5.-** Discontinuities in transients: Occur at times corresponding to lengths of Fermat paths.

**6.-** Two types of Fermat paths: Specular (parallel to surface normal) and boundary.

**7.-** Transients contain both Fermat and non-Fermat photons: Discontinuities from Fermat paths, continuous parts from non-Fermat paths.

**8.-** Identifying Fermat path lengths: Found at discontinuity locations in measured transients.

**9.-** Fermat paths depend only on geometry: Discontinuity locations in transients are independent of object reflectance (BRDF).

**10.-** Sphere constraint: Point on object lies on a sphere centered at virtual source with radius half the Fermat path length.

**11.-** Tangent sphere for specular paths: Sphere is tangent to the object surface.

**12.-** Fermat flow constraint: Spatial gradient of Fermat path length is parallel to virtual source and surface point direction.

**13.-** Reconstructing surface point: Intersect sphere with line parallel to gradient passing through virtual source.

**14.-** Specular paths provide surface normal: Gradient direction gives normal at reconstructed point.

**15.-** Estimating Fermat path gradients: Interpolated from Fermat path lengths at nearby virtual sources.

**16.-** Reconstruction pipeline: Scan virtual source, measure transients, detect discontinuities, estimate gradients, reconstruct oriented point cloud.

**17.-** Continuous surface from point cloud: Using algorithms like Poisson surface reconstruction.

**18.-** Applicable to different transient imaging systems: Demonstrated with SPAD+laser (picosecond) and OCT (femtosecond).

**19.-** Reconstructing various objects around a corner: Lambertian, semi-transparent, glossy, specular; convex and concave; millimeter accuracy.

**20.-** Reconstructing a coin: Both around a corner and through a diffuser using OCT; fine detail recovered.

Knowledge Vault built byDavid Vivancos 2024