Concept Graph & Resume using Claude 3 Opus | Chat GPT4 | Gemini Adv | Llama 3:

Resume:

1.-Richard Sutton is a famous contributor to machine learning, especially reinforcement learning. He wrote the well-known book "Reinforcement Learning and Interaction".

2.-Sutton believes representation learning is a key problem in AI/ML that is finally getting proper attention and hard work.

3.-Sutton wants to convince the audience that the key benefit of representation learning is enabling faster subsequent learning.

4.-Other potential benefits of representation learning include greater expressive power, better generalization, and producing intuitively pleasing representations.

5.-A show of hands reveals mixed opinions on the key benefit - some agree it's faster learning, others favor expressive power or generalization.

6.-Sutton argues representation learning requires a slow initial learning period in order to subsequently enable fast learning on new problems.

7.-This implies representation learning requires non-stationary, continual learning rather than one-time batch learning in order to demonstrate fast later learning.

8.-Sutton proposes a challenge problem called "JEFF" (generic online feature finding) to directly test ability to learn representations that enable fast learning.

9.-JEFF is an online regression problem with a two-layer target network where the goal is to find the hidden unit features.

10.-The hidden unit features are randomly generated when each JEFF instance is created. Finding them enables fast learning of the changing output.

11.-JEFF avoids test set leakage, has no role for unsupervised learning, is simple to implement, and directly tests fast learning ability.

12.-Sutton presents results on JEFF demonstrating the benefits of searching for good features vs a fixed random feature baseline.

13.-Combining feature search with gradient descent performs better than either alone, showing they both contribute to efficient feature finding.

14.-On non-stationary problems like a variant of MNIST with rotating labels, algorithms like backprop tend to do poorly and suffer catastrophic interference.

15.-A key to enabling fast learning and avoiding catastrophic interference appears to be adaptive per-feature learning rates that can preserve useful features.

16.-Sutton argues the field of representation learning has strayed from the original goal of enabling fast learning, but this should be the focus.

17.-Achieving this requires moving to online continual learning settings. JEFF provides a well-controlled way to study this without methodological issues.

18.-Sutton has preliminary results on parts of JEFF but not yet on the full non-stationary feature-finding problem. He recommends pursuing this.

19.-An audience member notes their own work found sequential learning can lead to faster learning, emergent from consolidation during simulated sleep.-

20.-Sutton agrees the rate of non-stationarity in JEFF could be varied, such as more slowly drifting changes rather than sudden shifts.

21.-Sutton likes the "prude directness" of sudden changes requiring fast adaptation, but agrees gradual changes are also worth considering.

22.-When asked about non-synthetic tasks with the desired properties, Sutton argues real life is full of repetitive but changing learning problems.

23.-Sutton resists the idea of explicitly signaling task changes to the learner, preferring the elegance of uninterrupted continual change.

24.-The shallow, two-layer formulation of JEFF is a necessary first step before considering deeper, hierarchical versions with features built from features.

25.-Sutton acknowledges JEFF as initially proposed doesn't involve hierarchical feature learning, but sees that as an important future direction to pursue.

26.-When features change at different rates, the representation learner should devote more learning resources to features that change more often.

27.-Sutton views finding features that enable fast learning as a new and unstudied problem of key importance that the field should pursue.

28.-Sutton apologizes for proposing JEFF without yet having complete results, but believes it is an important new research direction.

29.-An audience member suggests JEFF could be extended with a range of rates of change in different features.

30.-Sutton agrees, noting this occurs in the step-size adaptation results, and that learning should be allocated based on feature's rate of change.

Knowledge Vault built byDavid Vivancos 2024