Deep Learning of Graph Matching

Andrei Zanfir, Cristian Sminchisescu

**Concept Graph & Resume using Claude 3 Opus | Chat GPT4o | Llama 3:**

graph LR
classDef graph_matching fill:#f9d4d4, font-weight:bold, font-size:14px
classDef applications fill:#d4f9d4, font-weight:bold, font-size:14px
classDef affinity_matrix fill:#d4d4f9, font-weight:bold, font-size:14px
classDef deep_learning fill:#f9f9d4, font-weight:bold, font-size:14px
classDef challenges fill:#f9d4f9, font-weight:bold, font-size:14px
A[Deep Learning of

Graph Matching] --> B[Graph matching:

node correspondences. 1] A --> C[Applications: geometric

and semantic. 2] C --> D[Geometric matching:

same object,

different views.] C --> E[Semantic matching:

different objects,

same category.] A --> F[Affinity matrix: encodes

node similarities. 3] F --> G[Node similarities:

affinity matrix

encodes.] F --> H[Edge similarities:

affinity matrix

encodes.] A --> I[Relaxation: drops

constraints. 4] I --> J[Solution: leading

eigenvector.] A --> K[Deep learning:

deep features. 5] K --> L[Represents matrix

using features.] K --> M[Trains model

to predict truth.] A --> N[Challenges: gradients

and solver. 6] N --> O[Propagating gradients

through layers.] N --> P[Graph matching

solver.] A --> Q[Matrix backpropagation:

computes gradients. 7] Q --> R[Through matrix

functions.] A --> S[Factorization: reduces

complexity. 8] S --> T[Matrix operation

complexity reduced.] A --> U[Bi-stochastic layer:

mapping constraints. 9] U --> V[One-to-one

mapping.] A --> W[Loss function:

measures deviation. 10] W --> X[Deviation from

ground truth.] A --> Y[Results: competitive

performance. 11] Y --> Z[Sintel: geometric

dataset.] Y --> AA[PascalVOC/CUB:

semantic datasets.] A --> AB[Potential applications:

deep feature problems. 12] AB --> AC[Deep feature

hierarchies.] AB --> AD[Graph models:

text corpora,

social networks.] class A,B graph_matching class C,D,E applications class F,G,H affinity_matrix class I,J challenges class K,L,M deep_learning class N,O,P challenges class Q,R affinity_matrix class S,T challenges class U,V challenges class W,X challenges class Y,Z,AA results class AB,AC,AD potential_applications

Graph Matching] --> B[Graph matching:

node correspondences. 1] A --> C[Applications: geometric

and semantic. 2] C --> D[Geometric matching:

same object,

different views.] C --> E[Semantic matching:

different objects,

same category.] A --> F[Affinity matrix: encodes

node similarities. 3] F --> G[Node similarities:

affinity matrix

encodes.] F --> H[Edge similarities:

affinity matrix

encodes.] A --> I[Relaxation: drops

constraints. 4] I --> J[Solution: leading

eigenvector.] A --> K[Deep learning:

deep features. 5] K --> L[Represents matrix

using features.] K --> M[Trains model

to predict truth.] A --> N[Challenges: gradients

and solver. 6] N --> O[Propagating gradients

through layers.] N --> P[Graph matching

solver.] A --> Q[Matrix backpropagation:

computes gradients. 7] Q --> R[Through matrix

functions.] A --> S[Factorization: reduces

complexity. 8] S --> T[Matrix operation

complexity reduced.] A --> U[Bi-stochastic layer:

mapping constraints. 9] U --> V[One-to-one

mapping.] A --> W[Loss function:

measures deviation. 10] W --> X[Deviation from

ground truth.] A --> Y[Results: competitive

performance. 11] Y --> Z[Sintel: geometric

dataset.] Y --> AA[PascalVOC/CUB:

semantic datasets.] A --> AB[Potential applications:

deep feature problems. 12] AB --> AC[Deep feature

hierarchies.] AB --> AD[Graph models:

text corpora,

social networks.] class A,B graph_matching class C,D,E applications class F,G,H affinity_matrix class I,J challenges class K,L,M deep_learning class N,O,P challenges class Q,R affinity_matrix class S,T challenges class U,V challenges class W,X challenges class Y,Z,AA results class AB,AC,AD potential_applications

**Resume: **

**1.-** Graph matching: computing correspondences between nodes in two graphs under structural constraints.

**2.-** Applications: geometric matching (same object, different viewpoints) and semantic matching (different objects, same category).

**3.-** Affinity matrix: encodes node and edge similarities between graphs; used to find correspondences.

**4.-** Relaxation: drops binary and one-to-one constraints; solution is leading eigenvector of affinity matrix.

**5.-** Deep learning framework: represents affinity matrix using deep features; trains model to predict ground truth correspondences.

**6.-** Challenges: propagating gradients through matrix functional layers and graph matching solver.

**7.-** Matrix backpropagation: enables gradient computation through matrix functions in affinity matrix factorization.

**8.-** Factorization: reduces computational complexity of matrix operations from quartic to cubic/quadratic.

**9.-** Bi-stochastic operation layer: enforces one-to-one mapping constraints in final assignment matrix.

**10.-** Loss function: measures deviation between predicted and ground truth correspondences for training.

**11.-** Results: competitive performance on Sintel (geometric) and PascalVOC/CUB (semantic) datasets.

**12.-** Potential applications: problems involving deep feature hierarchies and graph models (e.g. text corpora, social networks).

Knowledge Vault built byDavid Vivancos 2024