The End Of Knowledge - Vault 6/60 - CVPR - 2020 - Open Challenges for Automated Machine Learning: Solving Intellectual Debt with Auto AI

graph LR classDef main fill:#f9d4f9, font-weight:bold, font-size:14px classDef basics fill:#f9d4d4, font-weight:bold, font-size:14px classDef applications fill:#d4f9d4, font-weight:bold, font-size:14px classDef challenges fill:#d4d4f9, font-weight:bold, font-size:14px classDef techniques fill:#f9f9d4, font-weight:bold, font-size:14px classDef future fill:#d4f9f9, font-weight:bold, font-size:14px Main[Open Challenges for
Automated Machine Learning:
Solving Intellectual Debt
with Auto AI] --> A[AI Basics
and Design] Main --> B[AI Applications] Main --> C[Challenges in
AI Systems] Main --> D[Techniques and
Solutions] Main --> E[Future Directions] A --> A1[AI today:
machine learning systems
design 1] A --> A2[Alexa: multiple
ML systems working
together 2] A --> A3[Alexa components
utilize machine learning 3] A --> A4[AI systems
involve large teams 4] A --> A5[Prime Air
uses ML-like control
algorithms 5] A --> A6[Service-oriented architecture
improves scalability, reliability 10] B --> B1[Supply chain
optimization uses machine
learning 8] B --> B2[ML predicts
demand for millions
of products 9] B --> B3[Amazons supply
chain: worlds largest
AI 24] B --> B4[ML used
in supply chain
management 25] B --> B5[Amazons re-architecting
led to AWS
development 26] B --> B6[Automated supply
chain creates instant
availability illusion 29] C --> C1[Separation of
concerns challenges system-wide
understanding 11] C --> C2[Intellectual debt:
understanding complex interacting
systems 12] C --> C3[Developing countries
face AI deployment
constraints 13] C --> C4[AutoML focuses
on components, ignores
interactions 18] C --> C5[Uganda biosurveillance
faces resource management
challenges 20] C --> C6[Unforeseen interactions
cause problems in
separated systems 27] D --> D1[Simulators predict
performance without real-world
testing 6] D --> D2[Multi-fidelity emulation
combines simulations with
data 7] D --> D3[Auto AI
automates functions in
complex systems 14] D --> D4[Emulation key
to managing complex
AI 15] D --> D5[Deep emulation
models subsystems and
interactions 16] D --> D6[Monitoring models,
predicting effects crucial
for ML 17] E --> E1[Data Science
Africa needs efficient,
automated AI 19] E --> E2[Local radio
monitoring uses keyword
detection 21] E --> E3[Safe AI
requires understanding system-wide
interactions 22] E --> E4[Technical and
intellectual debt important
in AI 23] E --> E5[Uganda crop
monitoring needs end-to-end
AI 28] E --> E6[Focus on
emulation for scalable
AI deployments 30] class Main main class A,A1,A2,A3,A4,A5,A6 basics class B,B1,B2,B3,B4,B5,B6 applications class C,C1,C2,C3,C4,C5,C6 challenges class D,D1,D2,D3,D4,D5,D6 techniques class E,E1,E2,E3,E4,E5,E6 future

Resume:

1.- AI today is primarily machine learning systems design, combining ML models to solve tasks.

2.- Alexa is an example of AI composed of multiple machine learning systems working together.

3.- Speech recognition, text-to-speech, and query processing in Alexa all utilize machine learning components.

4.- Amazon's AI systems involve large teams of people, including executives, scientists, and engineers.

5.- Prime Air delivery drones use control algorithms similar to machine learning techniques.

6.- Simulators and statistical emulators are used to predict system performance without real-world testing.

7.- Multi-fidelity emulation combines simulator predictions with real-world data to improve accuracy.

8.- Supply chain optimization uses machine learning to match product demand with supply.

9.- Forecasting teams use ML to predict demand for hundreds of millions of products weekly.

10.- Service-oriented architecture replaced monolithic code bases, improving scalability and reliability.

11.- Separation of concerns in software architecture can lead to challenges in understanding system-wide interactions.

12.- Intellectual debt refers to the challenge of understanding complex systems with many interacting components.

13.- Projects in developing countries face resource constraints in deploying AI systems compared to large tech companies.

14.- Auto AI aims to automate many functions currently performed manually in complex AI systems.

15.- Emulation is key to understanding and managing complex AI systems with multiple interacting components.

16.- Deep emulation involves using machine learning emulators to model different subsystems and their interactions.

17.- Monitoring models in production and predicting upstream/downstream effects is crucial for deployed ML systems.

18.- AutoML focuses on single ML components, but real-world systems require understanding of component interactions.

19.- Data Science Africa projects demonstrate the need for efficient, automated AI systems in resource-constrained environments.

20.- Biosurveillance and disease monitoring in Uganda face challenges due to limited resources for system management.

21.- Local radio monitoring uses keyword detection and snippet analysis to identify community issues.

22.- Safe and explainable AI deployment requires understanding of system-wide interactions and effects.

23.- Technical debt and intellectual debt are both important considerations in developing complex AI systems.

24.- The supply chain optimization system at Amazon is described as one of the world's largest AI systems.

25.- Machine learning is used in various aspects of supply chain management, including demand forecasting and inventory optimization.

26.- The re-architecting of Amazon's website led to the development of AWS and cloud services.

27.- Challenges arise when problems occur due to unforeseen interactions between separated system components.

28.- Mobile monitoring of crop diseases in Uganda demonstrates the need for end-to-end AI systems in resource-limited settings.

29.- Automated decision-making in supply chain management creates an illusion of instant product availability.

30.- The speaker advocates for the AI community to focus on sophisticated emulation techniques for scalable deployments.

Knowledge Vault built byDavid Vivancos 2024