Unsupervised Learning Of Equivariant Space-Time Capsules

Max Welling

**Concept Graph & Resume using Claude 3 Opus | Chat GPT4o | Llama 3:**

graph LR
classDef equivariance fill:#f9d4d4, font-weight:bold, font-size:14px
classDef symmetry fill:#d4f9d4, font-weight:bold, font-size:14px
classDef advantages fill:#d4d4f9, font-weight:bold, font-size:14px
classDef practical fill:#f9f9d4, font-weight:bold, font-size:14px
classDef applications fill:#f9d4f9, font-weight:bold, font-size:14px
classDef brain fill:#d4f9f9, font-weight:bold, font-size:14px
classDef quantum fill:#f9d4d4, font-weight:bold, font-size:14px
A[Unsupervised Learning Of

Equivariant Space-Time Capsules] --> B[Embed symmetries,

transform predictions 1] A --> C[Physics symmetries crucial 2] A --> D[Efficiency, sharing,

disentangling, depth 3] A --> E[Group representations,

generators, basis 4] A --> F[Convolutions on

spheres, graphs 5] F --> G[Handle neighborhoods,

rotation invariant 6] A --> H[Assemble molecules,

map numbers 7] A --> I[Disentangle pose

in capsules 8] A --> J[Remove dependencies,

topographic maps 9] J --> K[Brain maps

body, vision smoothly 10] J --> L[Wavelets decorrelate,

leave structure 11] A --> M[Infer latents,

amortized posteriors 12] M --> N[Decoder physics,

causal models 13] M --> O[Sparse variables,

correlated capsules 14] A --> P[Slow change,

sensory noise 15] P --> Q[Roll activations

in capsules 16] A --> R[Redundancy, disentangling,

topography, equivariance 17] R --> S[Presence norms,

pose patterns 18] A --> T[Continuous signals,

irregular sampling 19] T --> U[Gaussian processes,

uncertainty 20] T --> V[PDEs, sampling

grid invariance 21] A --> W[Oscillators,

measure states 22] W --> X[Quantum neural

networks 23] W --> Y[Particle excitations

in networks 24] W --> Z[Unsupervised capsules

resemble oscillators 25] A --> AA[Leverage dynamics,

temporal smoothness 26] A --> AB[Equivariance structures

networks 27] A --> AC[Topographic maps

inspire architectures 28] A --> AD[Quantum impact

via fields 29] A --> AE[Learn quantum

information theory 30] class A,B,R,S,AB equivariance class C symmetry class D,AA advantages class E,F,G,H,T,U,V practical class I,J,K,L,M,N,O,P,Q applications class AC,Z brain class W,X,Y,AD,AE quantum

Equivariant Space-Time Capsules] --> B[Embed symmetries,

transform predictions 1] A --> C[Physics symmetries crucial 2] A --> D[Efficiency, sharing,

disentangling, depth 3] A --> E[Group representations,

generators, basis 4] A --> F[Convolutions on

spheres, graphs 5] F --> G[Handle neighborhoods,

rotation invariant 6] A --> H[Assemble molecules,

map numbers 7] A --> I[Disentangle pose

in capsules 8] A --> J[Remove dependencies,

topographic maps 9] J --> K[Brain maps

body, vision smoothly 10] J --> L[Wavelets decorrelate,

leave structure 11] A --> M[Infer latents,

amortized posteriors 12] M --> N[Decoder physics,

causal models 13] M --> O[Sparse variables,

correlated capsules 14] A --> P[Slow change,

sensory noise 15] P --> Q[Roll activations

in capsules 16] A --> R[Redundancy, disentangling,

topography, equivariance 17] R --> S[Presence norms,

pose patterns 18] A --> T[Continuous signals,

irregular sampling 19] T --> U[Gaussian processes,

uncertainty 20] T --> V[PDEs, sampling

grid invariance 21] A --> W[Oscillators,

measure states 22] W --> X[Quantum neural

networks 23] W --> Y[Particle excitations

in networks 24] W --> Z[Unsupervised capsules

resemble oscillators 25] A --> AA[Leverage dynamics,

temporal smoothness 26] A --> AB[Equivariance structures

networks 27] A --> AC[Topographic maps

inspire architectures 28] A --> AD[Quantum impact

via fields 29] A --> AE[Learn quantum

information theory 30] class A,B,R,S,AB equivariance class C symmetry class D,AA advantages class E,F,G,H,T,U,V practical class I,J,K,L,M,N,O,P,Q applications class AC,Z brain class W,X,Y,AD,AE quantum

**Resume: **

**1.-** Equivariance: Neural networks can embed symmetries, allowing predictions to transform similarly to input transformations, inspired by physics.

**2.-** Symmetry in physics: Symmetries play a major role in physics theories like electromagnetism, general relativity, and elementary particles.

**3.-** Equivariance advantages: Equivariance enables data efficiency, parameter sharing, disentangling latent representations, and building deep networks.

**4.-** Practical equivariant basis: Specifying group representations and generators allows computing an equivariant basis, simplifying implementation.

**5.-** Equivariance on manifolds: Equivariance helps define convolutions on general manifolds like spheres and graphs.

**6.-** Graph neural networks: Equivariant graph neural networks handle arbitrary neighborhood sizes and are rotationally equivariant.

**7.-** Equivariant flows: Equivariant flows assemble molecules by mapping random numbers to atom types, interactions, and positions.

**8.-** Disentangling: Equivariance relates to disentangling, structuring latent spaces into relatively independent capsules encoding pose.

**9.-** Topographic ICA: Topographic independent component analysis removes higher-order dependencies by organizing filters in topographic maps.

**10.-** Brain topographic maps: The brain maps the body and visual orientations smoothly onto cortical areas.

**11.-** Higher-order dependencies: Wavelets decorrelate but leave structured dependencies in activation energy/volatility.

**12.-** Variational autoencoders: VAEs learn generative models by inferring latent variables using amortized approximate posteriors.

**13.-** Decoder physics: The VAE decoder can incorporate physical/causal world models, even simulators.

**14.-** Topographic VAE: Generates sparse latent variables from Gaussian activations and energies, correlated within capsules.

**15.-** Temporal coherence: Higher-order world concepts change slowly over time despite sensory noise.

**16.-** Sequence model: Energies are correlated across time by "rolling" activations forward in capsules.

**17.-** Statistical and equivariant concepts: Redundancy reduction, disentangling, topography, and equivariance are related ways to structure representations.

**18.-** Presence and pose: Object presence is encoded in capsule norms, pose in activation patterns.

**19.-** Continuous image signals: Images can be viewed as continuous signals irregularly sampled by pixels or superpixels.

**20.-** Gaussian process images: Gaussian processes turn discrete samples into continuous signals with uncertainty.

**21.-** PDEs and convolutions: Partial differential equations implement continuum limit convolutions invariant to sampling grids.

**22.-** Quantum field theory: Quantum fields are like oscillators at each point, measuring to sample entire states.

**23.-** Quantum neural networks: Neural networks can be formulated as quantum field theories implementable on quantum computers.

**24.-** Hinton particles: Quantum field formulation reveals "particle" excitations in neural networks.

**25.-** Equivariant capsules as oscillators: Learned unsupervised equivariant capsules resemble coupled oscillators.

**26.-** Dynamics and time: Neural networks should leverage dynamics and temporal smoothness priors reflecting the world.

**27.-** Mathematical structures: Deep mathematical theories like equivariance help structure neural networks.

**28.-** Brain inspiration: Neuroscience ideas like topographic maps inspire neural network architectures.

**29.-** Quantum computing potential: Quantum computers may significantly impact computer vision via quantum field formulations.

**30.-** Quantum information theory: Computer vision researchers can benefit from learning quantum information theory.

Knowledge Vault built byDavid Vivancos 2024