The End Of Knowledge - Vault 5/58 - CVPR - 2020 - TextureFusion: High-Quality Texture Acquisition for Real-Time RGB-D Scanning

graph LR classDef texture fill:#f9d4d4, font-weight:bold, font-size:14px classDef representation fill:#d4f9d4, font-weight:bold, font-size:14px classDef tile fill:#d4d4f9, font-weight:bold, font-size:14px classDef warp fill:#f9f9d4, font-weight:bold, font-size:14px classDef demo fill:#f9d4f9, font-weight:bold, font-size:14px A[TextureFusion: High-Quality Texture
Acquisition for Real-Time
RGB-D Scanning] --> B[Texture Fusion: Real-time
RGB-D scanning. 1] A --> C[Per-voxel color: Low
texture quality. 2, 4, 19] B --> D[Real-time 3D scanning:
Aligned, high-quality texture. 3] A --> E[Texture mapping: Separate
reconstruction, slow. 5] A --> F[2D texture topology
without meshing. 6] A --> G[Tile-based texture
data structure. 7, 23] G --> H[Texture to geometry:
Implicit surface projection. 8] G --> I[High texture resolution,
low voxel resolution. 9] A --> J[Simultaneous geometry and
texture updates. 10, 22] J --> K[Depth integration, texture
transfer to geometry. 11] J --> L[Camera pose estimation:
Depth map geometry. 12] J --> M[Camera motion field
for color/texture alignment. 13, 21] M --> N[Camera motion grid
optimization: Photometric consistency. 14] M --> O[Multi-level motion grid
optimization: Coarse to fine. 15] J --> P[Image warping, blending
for texture update. 16] A --> Q[Real-time RGB-D demo:
35 FPS. 17, 18] J --> R[Spatially varying warp:
High-quality texture integration. 20, 24] A --> S[Code and paper
available online. 25] class B,D texture class C,Q representation class F,G,H,I tile class M,N,O,P,R warp class E,J,K,L demo

Resume:

1.- Texture Fusion: High-quality texture acquisition for real-time RGB-D scanning.

2.- Existing real-time texture representation restores per-voxel color, resulting in low-quality texture.

3.- Texture Fusion enables real-time 3D scanning with aligned, high-quality texture.

4.- Traditional online scanning methods use per-voxel color representation, limiting texture quality and causing blurriness.

5.- Traditional texture mapping requires separate geometry and texture reconstruction, taking several minutes, unsuitable for real-time.

6.- Key insight: maintain 2D texture topology by avoiding explicit meshing.

7.- Tile-based texture data structure proposed for real-time 3D scanning.

8.- Tile tree-inspired structure maps texture to geometry by projecting implicit surface to assigned texture tile.

9.- Texture framework achieves high resolution even with low voxel resolution.

10.- Novel texture reconstruction framework simultaneously updates geometry and texture.

11.- Depth integrated into previous geometry, yielding current geometry; previous texture transferred to current geometry.

12.- Camera pose estimated using current depth map's geometric information due to asynchronous color/depth capture.

13.- Spatially varying camera motion field estimated to mitigate mismatch between current color image and texture map.

14.- Camera motion grid optimized by maximizing photometric consistency of local 3D texture patch with respect to image.

15.- Multi-level optimization of camera motion grid, from coarse to fine.

16.- Current image warped to align with previous texture map, then blended to update current texture map.

17.- Demonstrated application using conventional RGB-D camera, capturing 3D scenes with high-quality texture in real-time.

18.- Demo reconstructs geometry and texture simultaneously at 35 frames per second.

19.- Voxel representation has low-resolution texture limited by voxel resolution.

20.- Texture mapping without warping improves quality but still blurred.

21.- Spatially varying warp field integrates texture information with high quality.

22.- Real-time texture reconstruction framework presented for online RGB-D scanning.

23.- Tile-based texture data structure proposed for real-time texture acquisition.

24.- Spatially varying perspective warp introduced for real-time texture alignment.

25.- Code and paper available online.

Knowledge Vault built byDavid Vivancos 2024