Knowledge Vault 5 /34 - CVPR 2018
SPLATNet Sparse Lattice Networks for Point Cloud Processing
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz.
< Resume Image >

Concept Graph & Resume using Claude 3 Opus | Chat GPT4o | Llama 3:

graph LR classDef splatnet fill:#d4d4f9, font-weight:bold, font-size:14px classDef bcl fill:#f9d4d4, font-weight:bold, font-size:14px classDef performance fill:#d4f9d4, font-weight:bold, font-size:14px classDef features fill:#f9f9d4, font-weight:bold, font-size:14px A[SPLATNet Sparse Lattice
Networks for Point
Cloud Processing] --> B[SplatNet processes
3D point clouds directly. 1] A --> C[Standard CNNs unsuitable
for sparse point clouds. 2] C --> D[Voxelization, 2D projections
have limitations. 3] C --> E[SplatNet enables flexible
multi-scale filters. 4] B --> F[Jointly processes 2D images,
3D point clouds. 5] A --> G[Bilateral Convolution Layer
BCL is key. 6] G --> H[BCL filters sparse points
via interpolation. 7] H --> I[Permutohedral lattice scales better
higher dimensions. 8] G --> J[BCL has point features
and lattice features. 9] J --> K[Lattice features control
receptive field size. 10] G --> L[BCL allows different input-output
point locations. 11] L --> M[BCL variants project
between 2D and 3D. 12] A --> N[SplatNet-3D uses 1x1 convolutions,
BCL sequences. 13] N --> O[SplatNet-2D-3D adds
2D CNN branches. 14] N --> P[SplatNet-3D outperforms
on facade segmentation. 15] P --> Q[Joint SplatNet-2D-3D
improves prediction results. 16] P --> R[SplatNet matches state-of-the-art
on ShapeNet. 17] A --> S[XYZ+normals as 6D features
give flexibility. 18] A --> T[Efficient point cloud computation,
flexible fields. 19] T --> U[Code available online. 20] class A,B,C,D,E,F,N,O,P,Q,R,S,T,U splatnet class G,H,I,J,K,L,M bcl class P,Q,R performance class S features

Resume:

1.- SplatNet is a network architecture for processing 3D point clouds directly.

2.- Standard CNNs are not well-suited for sparse, unstructured point cloud data.

3.- Previous workarounds like voxelization or multi-view 2D projections have limitations.

4.- SplatNet enables flexible filter receptive fields to capture information at multiple scales.

5.- It also allows seamless joint processing of 2D image and 3D point cloud data.

6.- The key building block is Bilateral Convolution Layer (BCL).

7.- BCL efficiently filters sparse points via interpolation to/from vertices of a lattice structure.

8.- Permutohedral lattice scales better to higher dimensions than a regular grid.

9.- BCL specifies two feature sets: point features (what) and lattice features (where).

10.- Lattice features can use different scales to control receptive field size.

11.- BCL allows input and output points to be at different locations.

12.- Variants BCL-2D-to-3D and BCL-3D-to-2D project information between 2D and 3D.

13.- SplatNet-3D has 1x1 convolutions, a sequence of BCLs with increasing lattice scales, and concatenation.

14.- SplatNet-2D-3D adds 2D CNN branches with projection between 2D and 3D features.

15.- SplatNet-3D outperforms previous methods on Rimont 2014 building facade segmentation benchmark.

16.- The joint SplatNet-2D-3D further improves both 3D and 2D prediction results.

17.- On ShapeNet part segmentation, SplatNet matches state-of-the-art and joint 2D-3D helps further.

18.- Using XYZ+normals as 6D lattice features gives flexibility but only marginal gains here.

19.- The approach enables efficient point cloud computation, flexible receptive fields, and seamless 2D-3D integration.

20.- Code is available online.

Knowledge Vault built byDavid Vivancos 2024