Knowledge Vault 5 /38 - CVPR 2018
Squeeze-and-Excitation Networks
Jie Hu ; Li Shen ; Gang Sun
< Resume Image >

Concept Graph & Resume using Claude 3 Opus | Chat GPT4o | Llama 3:

graph LR classDef se fill:#f9d4d4, font-weight:bold, font-size:14px classDef cnn fill:#d4f9d4, font-weight:bold, font-size:14px classDef insights fill:#d4d4f9, font-weight:bold, font-size:14px classDef applications fill:#f9f9d4, font-weight:bold, font-size:14px classDef features fill:#f9d4f9, font-weight:bold, font-size:14px A[Squeeze-and-Excitation Networks] --> B[Squeeze-and-Excitation Networks:
ILSVRC 2017 winners 1] A --> C[Convolutional filters:
locally restricted representations 2] A --> D[SE block: dynamic
channel-wise recalibration 3] D --> E[SE insights: CNN
channel modeling limitations 4] D --> F[SE applications: network
pruning, compression 5] A --> G[Lower features: general
higher: task-specific 6] D --> H[SE block: global information,
improved learning 7] H --> I[SE captures distant
feature relationships 8] A --> J[Code, models
publicly available 9] A --> K[Q&A: transfer learning
applications 10] A --> L[Feature recalibration:
why it works 11] class B,D,H se class C cnn class E,F,K insights class G,I features class J,L applications

Resume:

1.- Squeeze-and-Excitation Networks won 1st place in ILSVRC 2017, reducing top-5 error to 2.251%

2.- Convolutional filters learn useful combinations of channel-wise and spatial information, but extraction is restricted to the local receptive field

3.- SE block improves representation power by dynamic channel-wise recalibration

4.- Provides insight into limitations of CNNs in modeling channel relationships

5.- Feature importance induced by SE block may be useful for tasks like network pruning for compression

6.- Lower-level features are more general, while higher-level features are more task-specific

7.- SE block embeds global information to increase receptive field and improve representation learning

8.- Allows capturing relationships between distant features (e.g. dog's mouth and tail)

9.- Code and models are publicly available

10.- Audience Q&A about potential applications to transfer learning

11.- Intuition behind why feature recalibration works

Knowledge Vault built byDavid Vivancos 2024