Knowledge Vault 5 /1 - CVPR 2015
What's Wrong with Deep Learning?
Yann LeCun
< Resume Image >

Concept Graph & Resume using Claude 3 Opus | Chat GPT4o | Llama 3:

graph LR classDef convNets fill:#f9d4d4, font-weight:bold, font-size:14px classDef theory fill:#d4f9d4, font-weight:bold, font-size:14px classDef unsupervised fill:#d4d4f9, font-weight:bold, font-size:14px classDef apps fill:#f9f9d4, font-weight:bold, font-size:14px A[Whats Wrong with
Deep Learning?] --> B[End-to-end learning integrates machine 1] A --> C[Neuroscience inspires hierarchical pattern
recognition 2] C --> D[LeCun pioneered convolutional nets
1980s-1990s 3] D --> E[Convolutional nets: character recognition,
segmentation 4] D --> F[Structural prediction on convolutional
nets 5] D --> G[Convolutional nets: object detection
1990s 6] D --> H[Convolutional nets extended semantic
segmentation 7] D --> I[ImageNet, GPUs: convolutional nets
breakthroughs 8] A --> J[Deep learning theory lacks
guidance 9] J --> K[Energy-based learning proposed for
reasoning 10] A --> L[Memory networks introduce memory
component 11] A --> M[Unsupervised learning challenges autoencoders 12] M --> N[What-where autoencoders couple convolutional,
deconvolutional 13] A --> O[Hardware implementations developing, some
misguided 14] class B,C,D,E,F,G,H,I convNets class J,K theory class L,M,N unsupervised class O apps


1.- Deep learning motivated by end-to-end learning, integrating entire machine into learning process.

2.- Hierarchical pattern recognition inspired by neuroscience, using convolutions, nonlinearities, and pooling.

3.- LeCun's early work on convolutional nets with local connections and shared weights in 1980s-1990s.

4.- Convolutional nets can be applied to entire images for character recognition and segmentation.

5.- Structural prediction on top of convolutional nets is an old idea, dating back to early 1990s.

6.- Convolutional nets used for object detection since 1990s, including face detection and pose estimation.

7.- Convolutional nets extended to semantic segmentation, integrating feature extraction, classification, and segmentation.

8.- ImageNet and fast GPUs led to breakthrough results in convolutional nets around 2012.

9.- Lack of theory in deep learning regarding architectures, number of layers, effective parameters, and local minima.

10.- Energy-based learning proposed as general approach for reasoning on top of deep learning.

11.- Memory networks introduce memory component into deep learning for tasks requiring remembering information.

12.- Unsupervised learning still a major challenge in deep learning; autoencoders a popular approach.

13.- What-where autoencoders couple convolutional and deconvolutional nets for unsupervised, supervised, or weakly supervised learning.

14.- Hardware implementations of convolutional nets being developed, but some approaches misguided.

Knowledge Vault built byDavid Vivancos 2024