The End Of Knowledge - Vault 2 - ICLR (2014-2023)

graph LR classDef robotics fill:#f9d4d4, font-weight:bold, font-size:14px; classDef perception fill:#d4f9d4, font-weight:bold, font-size:14px; classDef learning fill:#d4d4f9, font-weight:bold, font-size:14px; classDef sensorimotor fill:#f9f9d4, font-weight:bold, font-size:14px; classDef grasping fill:#f9d4f9, font-weight:bold, font-size:14px; A[Sergey Levine
ICLR 2016] --> B[Robotic learning insights apply
to machine learning. 1] A --> C[Perception, action coupling
simplifies both. 2] C --> D[Humans catch using gaze,
not physics models. 3] B --> E[Humanoid robots perform slowly,
separate stages. 4] C --> F[Humans perform skills smoothly,
combining perception, action. 5] B --> G[Algorithms for robots learning
sensorimotor skills presented. 6] G --> H[Guided policy search: instances
without vision, generalize with. 7] H --> I[Alternates trajectory optimization, supervised
training for complex policies. 8] G --> J[Experiments: guided search enables
complex sensorimotor skills. 9] J --> K[End-to-end policies outperform baselines,
show sensorimotor coupling benefit. 10] G --> L[Applied to manipulation, locomotion,
flight, other tasks. 11] B --> M[Scaling requires large, diverse data. 12] M --> N[Parallel robot setup collects
grasping data collectively. 13] N --> O[Grasping: continuous sensorimotor prediction
problem, not one-shot. 14] N --> P[Self-supervised learning based on
grasp success labels. 15] N --> Q[Neural network continuously selects
actions from images. 16] Q --> R[Continuous visual feedback outperformed
calibrated open-loop selection. 17] N --> S[Emergent behaviors: soft, heavy,
translucent object handling. 18] B --> T[Data diversity, real-world experience
critical for generalization. 19] T --> U[Robots could pool experience,
learn policies collectively. 20] class A,B,E,G,L,M,T robotics; class C,D,F perception; class H,I,J,K learning; class N,O,P,Q,R,S grasping; class U sensorimotor;

Resume:

1.-Deep robotic learning lessons apply broadly to machine learning systems interacting with the real world.

2.-Considering perception and action together as one sensorimotor loop can simplify both.

3.-Humans catch balls using gaze tracking to couple perception and action, not by explicitly modeling physics.

4.-Current humanoid robots perform tasks slowly using separate perception, modeling, planning and execution stages.

5.-Humans perform sensorimotor skills like opening doors smoothly by combining perception and action.

6.-The talk presents algorithms for robots to learn sensorimotor skills using deep neural networks trained end-to-end.

7.-Guided policy search breaks tasks into instances solvable without vision, then trains a deep network to generalize using vision.

8.-Guided policy search alternates trajectory optimization on simple models with supervised network training to handle complex policies and dynamics.

9.-Experiments show guided policy search enables robots to perform complex sensorimotor skills using vision.

10.-End-to-end visuomotor policies outperform two-stage baselines, showing the benefit of sensorimotor coupling.

11.-Guided policy search has been applied to manipulation, locomotion, flight and other tasks.

12.-Scaling up deep robotic learning requires large amounts of diverse data, similar to the success of supervised learning.

13.-A parallel robot setup was built to collect grasping data with 14 robots learning collectively.

14.-Grasping was posed as a continuous sensorimotor prediction problem rather than one-shot grasp selection.

15.-The parallel grasping system was trained with self-supervised learning based on grasp success labels.

16.-The system uses a neural network to continuously select grasping actions based on images without calibration.

17.-Continuous visual feedback for grasping outperformed a baseline using calibrated open-loop grasp selection.

18.-The grasping system exhibited interesting emergent behaviors like handling soft objects, heavy objects and translucent objects.

19.-Data diversity and real-world experience will be critical for highly generalizable sensorimotor skills in robotics.

20.-Robots could pool experience to collectively learn generalizable policies for repetitive real-world tasks like box packing.

Knowledge Vault built byDavid Vivancos 2024