The End Of Knowledge - Vault 6/45 - CVPR - 2019

graph LR classDef main fill:#f9d4f9, font-weight:bold, font-size:14px classDef learning fill:#f9d4d4, font-weight:bold, font-size:14px classDef models fill:#d4f9d4, font-weight:bold, font-size:14px classDef techniques fill:#d4d4f9, font-weight:bold, font-size:14px classDef challenges fill:#f9f9d4, font-weight:bold, font-size:14px classDef future fill:#d4f9f9, font-weight:bold, font-size:14px Main[Self-supervised learning] --> A[Learning Approaches] Main --> B[World Models] Main --> C[Techniques and Methods] Main --> D[Challenges and Limitations] Main --> E[Future Directions] A --> A1[Self-supervised learning: predict unlabeled input
parts 1] A --> A2[Supervised learning needs large labeled
datasets 2] A --> A3[Reinforcement learning requires many trials 3] A --> A4[Humans learn efficiently with few
samples 4] A --> A5[Babies learn concepts through world
interaction 6] A --> A6[Networks predict tasks on unlabeled
data 7] B --> B1[World models key to improving
AI 5] B --> B2[Model-based RL accelerates skill acquisition 16] B --> B3[Optimal control uses differentiable world
models 17] B --> B4[Driving simulation uses real data
model 18] B --> B5[Video prediction handles uncertainty with
latents 19] B --> B6[World model trains policy through
backpropagation 22] C --> C1[Energy functions capture data dependencies 10] C --> C2[Contrastive methods push data energy
down 11] C --> C3[Regularizing latents limits low-energy space 12] C --> C4[Sparse coding learns data representations 13] C --> C5[Predictor estimates optimal sparse codes 14] C --> C6[Dropout prevents latents capturing too
much 20] D --> D1[Continuous, high-dimensional data prediction challenge 8] D --> D2[Latent variables model uncertainty, multiple
outputs 9] D --> D3[Multi-layer sparse codes ongoing research 15] D --> D4[Inverse curiosity avoids unreliable predictions 23] D --> D5[Uncertainty in continuous spaces key
challenge 24] D --> D6[GANs useful but difficult to
train 25] E --> E1[Adding noise limits information, like
VAEs 21] E --> E2[Seeking reliable uncertainty learning methods 26] E --> E3[Model-based RL interest renewed despite
limitations 27] E --> E4[Classification avoids uncertainty in self-supervision 28] E --> E5[Jigsaw puzzle predicts image patch
positions 29] E --> E6[Address continuous, high-dimensional uncertainty learning 30] class Main main class A,A1,A2,A3,A4,A5,A6 learning class B,B1,B2,B3,B4,B5,B6 models class C,C1,C2,C3,C4,C5,C6 techniques class D,D1,D2,D3,D4,D5,D6 challenges class E,E1,E2,E3,E4,E5,E6 future

Resume:

1.- Self-supervised learning: Learning from unlabeled data by predicting parts of the input from other parts, without human-curated labels.

2.- Supervised learning limitations: Requires large labeled datasets, which aren't always available for all problems.

3.- Reinforcement learning inefficiency: Requires many trials, impractical for real-world applications like self-driving cars.

4.- Human learning efficiency: Humans learn quickly with few samples, suggesting we're missing something in machine learning approaches.

5.- Learning world models: Key to improving AI is learning models of how the world works to enable efficient learning.

6.- Baby learning stages: Babies learn basic concepts like object permanence and gravity over time through world interaction.

7.- Self-supervised learning definition: Training large networks to understand the world through prediction tasks on unlabeled data.

8.- Prediction under uncertainty: Challenge in self-supervised learning for continuous, high-dimensional data like images or video.

9.- Latent variable models: Using additional variables to model uncertainty and generate multiple possible outputs.

10.- Energy-based learning: Formulating self-supervised learning as learning energy functions that capture data dependencies.

11.- Contrastive methods: Pushing down energy of data points while pushing up energy of points outside the data manifold.

12.- Regularized latent variables: Limiting the volume of low-energy space by regularizing latent variables.

13.- Sparse coding: Early example of regularized latent variable systems for learning data representations.

14.- Predictive sparse decomposition: Training a predictor to estimate optimal sparse codes, avoiding expensive optimization.

15.- Hierarchical representations: Learning multi-layer sparse codes for complex data like images, an ongoing research area.

16.- Model-based reinforcement learning: Using learned world models to accelerate skill acquisition, especially in motor tasks.

17.- Optimal control theory: Classical approach to control using differentiable world models, basis for some modern AI techniques.

18.- Autonomous driving example: Learning to drive in simulation using a world model trained on real driving data.

19.- Video prediction model: Neural network trained to predict future frames in driving videos, handling uncertainty with latent variables.

20.- Regularizing latent variables: Using techniques like dropout to prevent latent variables from capturing too much information.

21.- Variational autoencoder similarity: Adding noise to encoder output to limit information content, similar to VAEs.

22.- Policy network training: Using the learned world model to train a driving policy through backpropagation, without real-world interaction.

23.- Inverse curiosity: Encouraging the agent to stay in areas where its world model is accurate, avoiding unreliable predictions.

24.- Handling uncertainty in continuous spaces: Key technical challenge in self-supervised learning for high-dimensional data.

25.- GAN limitations: While useful for handling uncertainty, GANs are difficult to train reliably.

26.- Alternatives to GANs: Seeking more reliable methods for learning under uncertainty in high-dimensional spaces.

27.- Model-based RL resurgence: Renewed interest in model-based reinforcement learning, despite previous theoretical limitations.

28.- Classification workaround: Some self-supervised methods avoid uncertainty by turning prediction into classification tasks.

29.- Jigsaw puzzle example: Self-supervised task of predicting relative positions of image patches, avoiding pixel-level prediction.

30.- Future direction: Need to directly address the problem of learning under uncertainty in continuous, high-dimensional spaces.

Knowledge Vault built byDavid Vivancos 2024