The End Of Knowledge - Vault 6/52 - CVPR - 2020 - Human and Machine Learning for Assistive Autonomy

graph LR classDef main fill:#f9d4f9, font-weight:bold, font-size:14px classDef basics fill:#f9d4d4, font-weight:bold, font-size:14px classDef shared fill:#d4f9d4, font-weight:bold, font-size:14px classDef custom fill:#d4d4f9, font-weight:bold, font-size:14px classDef safety fill:#f9f9d4, font-weight:bold, font-size:14px classDef future fill:#d4f9f9, font-weight:bold, font-size:14px Main[Human and Machine
Learning for Assistive
Autonomy] --> A[Rehabilitation and
Assistive Devices] Main --> B[Shared Control
Concepts] Main --> C[Customization and
User Preferences] Main --> D[Safety and
Autonomy] Main --> E[Future Directions] A --> A1[Rehabilitation machines
restore function, capturing
control signals 1] A --> A2[Higher amputations:
fewer muscles, more
complex control 2] A --> A3[Power wheelchairs:
various control interfaces 3] A --> A4[Doorway navigation
challenging with limited
interfaces 4] A --> A5[Robotic arms
require 6D control,
use modes 5] A --> A6[Limited interfaces
hinder robotic arm
operation 6] B --> B1[AI assists,
customizing autonomy level
critical 7] B --> B2[Shared control
blends human and
machine control 8] B --> B3[Task metrics
similar, user preferences
vary 9] B --> B4[Teleoperation: task-level
commands differ from
control signals 20] B --> B5[Graphical model
represents sip-and-puff command
transitions 21] B --> B6[Filtering inferred
unintended commands improves
task success 22] C --> C1[Preferences change
with interface, customization
needed 10] C --> C2[Users verbally
customized control-autonomy blending
function 11] C --> C3[Customized control
enabled smooth 6D
operation 12] C --> C4[Customization eliminated
performance differences between
groups 13] C --> C5[Users consider
factors beyond standard
metrics 14] C --> C6[Adapting autonomy:
feedback, filtering, trust,
temporality 15] D --> D1[Safety-aware shared
control rejects unsafe
commands 23] D --> D2[Shared control
learns human-robot system
models 24] D --> D3[Safety-aware control
improves demonstration quality,
enables completion 25] D --> D4[Learned controller
from shared control
reproduces task 26] D --> D5[Body-machine interface
maps motions to
control signals 27] D --> D6[Gradual autonomy
engagement aims to
bootstrap learning 28] E --> E1[Motor impairments
complicate robot demonstrations,
studies needed 16] E --> E2[Baseline tasks:
wheelchair course, assessments,
VR tests 17] E --> E3[Command success
differed between groups,
not time 18] E --> E4[Daily usage
affected response time,
not success 19] E --> E5[Pilot determines
human-autonomy co-adaptation rates 29] E --> E6[Key factors:
feedback, filtering, trust,
temporal/co-adaptation 30] class Main main class A,A1,A2,A3,A4,A5,A6 basics class B,B1,B2,B3,B4,B5,B6 shared class C,C1,C2,C3,C4,C5,C6 custom class D,D1,D2,D3,D4,D5,D6 safety class E,E1,E2,E3,E4,E5,E6 future

Resume:

1.- Rehabilitation uses machines to restore lost function and assist with gaps from injury/disease. Capturing control signals is a key challenge.

2.- Higher amputation levels have fewer residual muscles for EMG control of prosthetic arms, yet require controlling more complex movements.

3.- Power wheelchairs are commonly used assistive machines. Control interfaces range from proportional joysticks to non-proportional switches like sip-and-puff.

4.- Navigating a wheelchair through a doorway is challenging even for experts, especially with limited interfaces like a head array.

5.- Robotic arms require 6-dimensional control of position and orientation. Interfaces can't provide this simultaneously, so control modes are used.

6.- People with the most limited interfaces who would benefit most from robotic arms have the hardest time operating them.

7.- Adding sensors and AI to make assistive machines into assistive robots can help, but customizing level of autonomy is critical.

8.- Shared control paradigms blend human and machine control in various ways. Studies compare autonomy to shared control and teleoperation.

9.- When comparing shared control paradigms, task metrics don't vary much but user preferences do. Providing options is important.

10.- Preferences change when the control interface changes, even with the same autonomy. Customization to the individual is needed.

11.- An exploratory study let users verbally customize a piecewise linear function blending their control with the robotic arm's autonomy.

12.- The customized shared control eliminated mode switches and enabled smooth 6D control compared to unassisted teleoperation for the user.

13.- The customized paradigm eliminated performance differences between participants with spinal cord injuries and uninjured participants seen with other paradigms.

14.- User customizations considered factors besides standard robot metrics like minimizing time and effort. Users have useful insights to capture.

15.- Challenges in adapting autonomy to users include: type of feedback signal, information filtering by impairment/interface, trusting the human, and temporality.

16.- Providing demonstrations to robots is hard for people with motor impairments. Large-scale teleoperation studies aim to characterize usage.

17.- Wheelchair obstacle course, clinical assessment, and VR command following and trajectory following tasks were used as baselines.

18.- Command following success differed between subject groups, but not response time. Interface affected response time but not success.

19.- Daily interface usage (expertise) affected response time but not success in spinally-injured users, so adaptation over time is expected.

20.- Teleoperation involves task-level human commands that don't always match the actual control signals issued due to the interface layer.

21.- A graphical model represents transitions between intended and measured sip-and-puff commands for a robot. Data informs intended input distributions.

22.- Filtering or correcting inferred unintended commands based on the model enables task success and reduces mode switches in a pilot.

23.- Task-agnostic safety-aware shared control analyzes system safety and rejects or overrides unsafe human commands while maintaining human control.

24.- The shared control learns models of the joint human-robot system to inform its autonomous interventions when needed for safety.

25.- Safety-aware shared control enables task completion and improved demonstration quality for learning compared to unassisted human control.

26.- A learned autonomous controller from shared control demos can reproduce the task, addressing covariance shift, unlike learning from unassisted demos.

27.- A body-machine interface maps high-dimensional residual body motions to low-dimensional control signals, traditionally with PCA, for various applications.

28.- Gradually engaging and fading robot autonomy aims to bootstrap human motor learning to achieve high-DoF control with limited body motions.

29.- A pilot will determine rates of co-adaptation between the human and autonomy to parameterize a full 20-participant, 20-session study.

30.- Key factors in devising suitably adaptive assistive shared autonomy are feedback, information filtering, trust, temporal adaptation, and co-adaptation.

Knowledge Vault built byDavid Vivancos 2024