The End Of Knowledge - Vault 2 - ICLR (2014-2023)

graph LR classDef AI fill:#f9d4d4,stroke-width:2px classDef market fill:#d4f9d4,stroke-width:2px classDef research fill:#d4d4f9,stroke-width:2px classDef ray fill:#f9f9d4,stroke-width:2px classDef future fill:#f9d4f9,stroke-width:2px A[Michael I. Jordan
ICLR 2020] --> B[AI: planetary-scale
human-computer systems 1] B --> C[Building systems
analogous to engineering 2] A --> D[1990s: ML emerged
in real-world 3] A --> E[Current: pattern recognition
with NNs 4] A --> F[Next: planetary decision
networks, markets 5] F --> G[Decisions need error
bars, experiments 6] F --> H[Markets: decentralized
decision algorithms 7] H --> I[Markets handle load
balancing naturally 8] H --> J[Two-way producer-consumer
markets attack scarcity 9] A --> K[Research: multi-way exploration,
UCB, FDR 10] K --> L[Multi-armed bandits
with competition 11] K --> M[UCB exploration improves
Q-learning regret 12] K --> N[FDR important in
multiple testing 13] N --> O[Online FDR algorithms
control errors 14] A --> P[Goal: distributed ML
for decisions 15] P --> Q[Ray: laptop to
cloud ML 16] Q --> R[Ray: distributed Python
tasks, actors 17] Q --> S[Ray used for
rapid recommenders 18] A --> T[Pattern recognition
not enough 19] A --> U[Future ML needs
economics, markets 20] class A,B,E AI class F,H,I,J market class K,L,M,N,O research class P,Q,R,S ray class T,U future

Resume:

1.-The speaker sees AI as creating planetary-scale systems that involve humans, computers, data flows, decisions, learning and personalization to help humans at scale.

2.-Building these systems is analogous to developing fields like chemical and electrical engineering to make chemistry and electricity work at scale in the real world.

3.-Machine learning emerged as a real-world phenomenon in the 1990s with systems like Amazon's backend for fraud detection, supply chain management, and recommendations.

4.-The current era is focused on pattern recognition with neural networks, enabling speech recognition, computer vision and translation, but it's just one component.

5.-Emerging next are large-scale networks of decisions and data flows at a planetary scale, involving concepts from game theory, market design and economics.

6.-Real-world decisions require error bars, what-if experiments, provenance, dialogue, not just thresholding a single prediction, no matter how good the learning system is.

7.-Markets can serve as decentralized decision algorithms that are adaptive, robust, scalable with long lifetimes, analogous to intelligence. Research opportunities emerge combining microeconomics and machine learning.

8.-Load balancing issues arise when recommending the same item (e.g. restaurant, stock, route) to too many people. Markets can naturally handle this.

9.-Producers and consumers should be linked in two-way markets based on data analysis to attack scarcity, create jobs and wealth in domains like music.

10.-Research directions include multi-way exploration markets, adding UCB exploration to reinforcement learning, and controlling false discovery rates in networked asynchronous decision making.

11.-In multi-armed bandits with competition, agents may pick popular arms less due to conflict, requiring game theory. Logarithmic regret is still possible.

12.-Adding UCB exploration to Q-learning leads to improved square root regret bounds compared to epsilon-greedy exploration, connecting to model-based RL.

13.-In multiple hypothesis testing, controlling the false discovery rate (FDR) is important since most surprising results amplified tend to be the false ones.

14.-Online FDR algorithms exist that control FDR at any time by adaptively allocating an error budget as a function of past discoveries.

15.-The speaker aims to build distributed ML systems for complex real-world decision making, like an improved worldwide medical system to respond to pandemics.

16.-Ray is a distributed system for decision-focused ML that runs from a laptop to the cloud, unifying libraries for training, serving, RL, etc.

17.-Ray provides distributed tasks (functions) and actors (objects) in Python, allowing code to easily scale with minimal changes. Performance reaches 1M tasks/sec.

18.-Ray is open-source on GitHub, used in production by companies for applications like rapid recommender system updating, improving click-through rates.

19.-While current ML focused on pattern recognition is remarkable, it is a commodity and not enough for consequential decisions that require explanations and dialogue.

20.-Future ML systems must incorporate ideas from economics and markets for large-scale decision making. New problems and mechanisms spanning ML and economics will emerge.

Knowledge Vault built byDavid Vivancos 2024