The End Of Knowledge - Vault 6/47 - CVPR - 2019

graph LR classDef main fill:#f9d4f9, font-weight:bold, font-size:14px classDef open fill:#f9d4d4, font-weight:bold, font-size:14px classDef adversarial fill:#d4f9d4, font-weight:bold, font-size:14px classDef dataset fill:#d4d4f9, font-weight:bold, font-size:14px classDef stability fill:#f9f9d4, font-weight:bold, font-size:14px classDef future fill:#d4f9f9, font-weight:bold, font-size:14px Main[The Deep Unknown] --> A[Open Set and
Robustness] Main --> B[Adversarial Examples] Main --> C[Dataset Shifts] Main --> D[Stability and Invariance] Main --> E[Future Directions] A --> A1[Open category detection recognizes novel
classes 1] A --> A2[Open set balances empirical, open
risks 3] A --> A3[Robustness, out-of-distribution detection remain
challenges 9] A --> A4[Humans more robust to adversarial
perturbations 10] A --> A5[Dynamic environments induce problematic dataset
shifts 13] A --> A6[Dataset shifts: population, policy, measurement
changes 14] B --> B1[Adversarial examples problem for deep
learning 2] B --> B2[Adversarial inputs change models prediction
drastically 4] B --> B3[Internal representations vulnerable to adversarial
attacks 5] B --> B4[Adversarial vulnerability cause remains unclear 6] B --> B5[Minor changes correct natural misclassifications 7] B --> B6[ReLU infinity may cause adversarial
vulnerability 8] C --> C1[Proactive approaches preferred for costly
mistakes 15] C --> C2[Graphical models represent arbitrary dataset
shifts 16] C --> C3[Learn models invariant to specified
shifts 17] C --> C4[Hierarchy of shift-stable distributions exists 18] C --> C5[Algorithm finds shift-invariant stable predictor 19] C --> C6[Estimate, combine conditional distributions for
stability 20] D --> D1[Procedure: sound, complete, optimal for
invariance 21] D --> D2[Unobserved confounding prevents finding stable
predictor 22] D --> D3[Pneumonia example: hospital-invariant prediction using
radiographs 23] D --> D4[Optimal stable predictor improves on
pruning 24] D --> D5[Counterfactual distributions provide better stable
predictor 25] D --> D6[Estimating counterfactuals is open problem 26] E --> E1[Causality, generative models may improve
robustness 11] E --> E2[Democratize understanding of ML reliability,
robustness 12] E --> E3[Understanding domain, data, failures is
critical 27] E --> E4[Mixed approach for risk mitigation 28] E --> E5[Reliability engineering principles for machine
learning 29] E --> E6[Mindset shift needed for real-world
deployment 30] class Main main class A,A1,A2,A3,A4,A5,A6 open class B,B1,B2,B3,B4,B5,B6 adversarial class C,C1,C2,C3,C4,C5,C6 dataset class D,D1,D2,D3,D4,D5,D6 stability class E,E1,E2,E3,E4,E5,E6 future

Resume:

1.- Open category detection is important for machine learning systems to recognize novel classes not seen during training.

2.- Adversarial examples are a problem for deep learning models. The underlying causes are still being investigated.

3.- Open set recognition requires balancing empirical risk and open space risk, the risk of labeling unknown inputs as known classes.

4.- Adversarial examples are imperceptibly close to real inputs but drastically change the model's prediction. They may be inevitable in high dimensions.

5.- Adversarial examples can be generated by attacking the model's internal representations, not just the output layer. This makes them more transferable.

6.- The underlying cause of adversarial vulnerability is still unclear. Hypotheses like aliasing or reliance on unstable features have been ruled out.

7.- Adversarial examples occur naturally, not just artificially. Minor input changes can correct misclassified natural images.

8.- ReLU activations going towards infinity in open space may contribute to adversarial vulnerability. Tent activations which go up then down may help.

9.- Robustness to distribution shift and out-of-distribution detection are important challenges as machine learning systems are deployed in the real world.

10.- Humans are much more robust to adversarial perturbations than current deep learning models. Mimicking human visual processing could improve model robustness.

11.- Causality and building generative models of the world that capture abstract invariances may help achieve human-like robustness in machine learning systems.

12.- As more people use machine learning, we need to democratize not just the tools but also understanding of reliability and robustness.

13.- Machine learning systems are often deployed in dynamic environments that can induce problematic dataset shifts degrading performance.

14.- Examples of problematic dataset shifts include changes in population, treatment policies, measurement devices, and more.

15.- Reactive approaches to dataset shift like continually adapting the model are problematic when mistakes are costly. Proactive approaches are preferred.

16.- Graphical models provide an intuitive framework for representing arbitrary dataset shifts in terms of unstable pathways between variables.

17.- The goal is to learn models that are invariant to pre-specified problematic shifts. Simply using stable features is too conservative.

18.- There is a hierarchy of shift-stable distributions, from graph pruning to interventional to counterfactual, capturing more stable information.

19.- An algorithm is presented that takes data and a causal graph specifying shifts and returns a stable predictor invariant to those shifts.

20.- The algorithm tells you what conditional distributions to estimate from data and combine to obtain a stable interventional distribution.

21.- The procedure is sound (guaranteed invariance to specified shifts), complete (if it fails, no other invariant predictor exists), and optimal.

22.- Unobserved confounding can prevent finding a stable predictor. Measuring confounders or relaxing invariance requirements are potential solutions.

23.- A pneumonia prediction example illustrates learning a predictor invariant to hospital department by conditioning on stable radiograph features.

24.- The optimal stable interventional predictor improves on graph pruning by capturing more stable information, but may still be suboptimal.

25.- Counterfactual distributions can provide an even better stable predictor by keeping more non-problematic pathways, but are harder to estimate.

26.- Estimating counterfactual distributions involves fixing problematic variables and inferring counterfactual states of others, which is an open problem.

27.- To build reliable machine learning systems, deeply understanding the domain, data, and potential failure modes is critical.

28.- Deploying machine learning requires a mixed qualitative and quantitative approach to proactively reason about and mitigate risks.

29.- Key principles from reliability engineering for machine learning are failure prevention, reliability monitoring, and maintaining deployed systems.

30.- A shift in mindset is needed in machine learning research and practice to seriously engage with these issues as real-world deployment increases.

Knowledge Vault built byDavid Vivancos 2024