Knowledge Vault 2/55 - ICLR 2014-2023
Pierre-Yves Oudeyer ICLR 2019 - Invited Talk - Developmental Autonomous Learning: AI, Cognitive Sciences and Educational Technology
<Resume Image >

Concept Graph & Resume using Claude 3 Opus | Chat GPT4 | Gemini Adv | Llama 3:

graph LR classDef learning fill:#f9d4d4, font-weight:bold, font-size:14px; classDef development fill:#d4f9d4, font-weight:bold, font-size:14px; classDef exploration fill:#d4d4f9, font-weight:bold, font-size:14px; classDef applications fill:#f9f9d4, font-weight:bold, font-size:14px; classDef future fill:#f9d4f9, font-weight:bold, font-size:14px; A[Pierre-Yves Oudeyer
ICLR 2019] --> B[Children: extraordinary autonomous learners. 1] A --> C[Developmental robotics
studies skill learning. 2] C --> D[Algorithmic models understand,
apply development. 3] C --> E[Developmental forces: morphology
, biases, motivation. 4] A --> F[Children's curiosity-driven
exploration. 5] F --> G[Child explores for
predictive world models. 6] F --> H[Learning progress hypothesis
for interestingness. 7] A --> I[Robotic playgrounds model
high-dimensional discoveries. 8] I --> J[Intrinsic motivation ingredients:
primitives, memory. 9] I --> K[Interestingness measures:
forward/inverse model progress. 10] I --> L[Prediction experiments sample
high progress regions. 11] I --> M[Splitting spaces, monitoring
progress enables learning. 12] I --> N[Modular exploration
efficiently controls objects. 13] I --> O[Goal exploration outperforms
forward model. 14] O --> P[Goal exploration produces
diverse effects. 15] I --> Q[Learning goal spaces from
pixels challenging. 16] A --> R[Curiosity-driven vocal
development model. 17] R --> S[Model replicates infant
vocalization stages. 18] R --> T[Multi-agent simulations
yield language distributions. 19] A --> U[Human exploration focuses on
intermediate complexity. 20] A --> V[Curiosity algorithms generate
personalized curricula. 21] V --> W[System tracks progress,
proposes optimal exercises. 22] V --> X[Algorithm variations increase
motivation vs expert-designed. 23] A --> Y[Spontaneous exploration's role
needs further study. 24] A --> Z[Goal exploration enables
efficient robot learning. 25] Z --> AA[Mechanisms model development,
tool use, language. 26] A --> AB[Curiosity improves educational
technologies. 27] A --> AC[Speech communication bootstraps
without agent models. 28] A --> AD[Inverse models rely on
learned representations. 29] A --> AE[Learning representations for
curiosity is challenging. 30] class B,C,D,E development; class F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,Y exploration; class V,W,X,AB applications; class Z,AA,AC,AD,AE future;


1.-Children are extraordinary learners who efficiently acquire everyday skills autonomously and developmentally, without external engineers retuning hyperparameters for each new task.

2.-Developmental robotics studies how developmental structures form and their potential role in enabling skill learning under time, energy and computation limits.

3.-Researchers develop algorithmic models of human development to understand it, apply insights to AI/ML for more flexible machines, and develop educational technologies.

4.-Body morphology, cognitive biases, social guidance, and intrinsic motivation are developmental forces studied that act as guidances or constraints over learning.

5.-Children engage in curiosity-driven exploration, being intrinsically interested in novelty, cognitive dissonance, surprise and optimal challenge, as discussed by psychologists since the 1940s-50s.

6.-A framework views the child as a sense-making organism exploring to make predictive world models and control the world via intrinsically motivated experiments.

7.-The learning progress hypothesis proposes interestingness of an experiment is proportional to the amount of change in prediction or goal achievement errors.

8.-Researchers use robotic playgrounds as modeling tools to study mechanisms enabling children's discoveries in high-dimensional environments, like discovering affordances and speech communication.

9.-Intrinsic motivation systems use ingredients like dynamic movement primitives, object-based perception, self-supervised forward/inverse model learning, episodic memory, and autonomous learning curriculum organization.

10.-Two proposed ways to measure interestingness for exploration are learning progress in forward models to choose policy parameters, or in inverse models for goals.

11.-Prediction experiments sample policy parameters in forward model regions with high learning prediction progress. Goal exploration samples target descriptors maximizing competence progress.

12.-Splitting parameter/goal spaces into regions, monitoring learning progress in each, and focusing exploration on high progress regions enables efficient learning in high dimensions.

13.-Modular goal exploration in robots efficiently learns to discover and control different objects by focusing on those providing maximal learning progress.

14.-Curiosity-driven exploration of a forward model is less efficient than goal exploration due to redundancy and inhomogeneities typical in robotic spaces.

15.-Goal exploration incentivizes being good on average at producing diverse effects, while forward model exploration may only learn many ways to produce few effects.

16.-Learning disentangled goal space representations from pixels is possible using beta-VAE but more difficult in realistic robotic setups than using object-based representations.

17.-A vocal development model using curiosity-driven learning reproduces the developmental stages of infant vocalizations, including the shift from self-exploration to imitation.

18.-Developmental trajectories emerging from the model exhibit both regularity in typical stage order and diversity in individual differences, matching child development.

19.-Multi-agent simulations with intrinsically motivated vocal learners playing language games lead to synchronized emergent communication systems mapping well to world language distributions.

20.-New human behavioral paradigms study free exploration across many tasks, finding subjects focus on levels of intermediate complexity corresponding to maximal learning progress.

21.-Curiosity-driven algorithms can generate personalized curricula for human learners that maximize learning efficiency and motivation, such as for primary school mathematics.

22.-For each student, the system tracks learning progress across exercise properties and incrementally proposes exercises maximizing progress, outperforming expert-designed fixed progressions.

23.-Variations of the algorithm result in children being more intrinsically motivated about learning mathematics compared to an expert-designed "oracle" algorithm.

24.-Spontaneous exploration plays a fundamental role in development that is not yet well understood or studied in machine learning but presents an important future path.

25.-Autonomous goal exploration and learning progress enable real-world robots to efficiently learn complex skills in high dimensions by self-organizing developmental trajectories.

26.-The same mechanisms can model the self-organization of infant development, discovery of tool use, and aspects of language acquisition.

27.-Educational technologies can be improved by applying curiosity-driven algorithms to generate personalized curricula adapted to each learner's progress across multiple dimensions.

28.-Basic speech communication can bootstrap without initially requiring mental models of others as agents, which may emerge more easily after interaction structures develop.

29.-Learning inverse models for goal exploration currently relies on state representations and similarity metrics, which can be learned in embodied agents leveraging time-correlation structure.

30.-Ongoing work focuses on learning distance functions and representations that enable curiosity-driven exploration and learning without externally provided goal spaces, a key challenge.

Knowledge Vault built byDavid Vivancos 2024