The End Of Knowledge - Vault 3 - G.TEC BCI & Neurotechnology Spring School 2024 - Day 5

graph LR classDef synergy fill:#f9d4d4, font-weight:bold, font-size:14px; classDef applications fill:#d4f9d4, font-weight:bold, font-size:14px; classDef research fill:#d4d4f9, font-weight:bold, font-size:14px; classDef future fill:#f9f9d4, font-weight:bold, font-size:14px; classDef acknowledgments fill:#f9d4f9, font-weight:bold, font-size:14px; A[Ramana Kumar Vinjamuri] --> B[ synergy-based
brain-machine interfaces. 1] A --> C[Goal: understand brain's control
of dexterous hand. 2] C --> D[Top-down: neural representations
of hand movements. 3] C --> E[Bottom-up: control mechanisms
of hand movements. 3] C --> F[Applications: motor control, rehab,
skill learning. 4] A --> G[Synergies: building blocks of
shared movements. 5] G --> H[Part 1: dimensionality reduction,
behavior to brain. 6] H --> I[Synergies extracted from kinematics
using PCA. 7] I --> J[Linear methods capture global
movement patterns. 8] I --> K[Fusing kinematics, muscle activity
improves representation, reconstruction. 8] H --> L[Dynamical synergies inform population
activation, pressure points. 9] G --> M[Part 2: neural representations,
brain to behavior. 10] M --> N[EEG decodes synergy-based hand
movements. 90%+ accuracy. 11] N --> O[Extending to individuals with
paralysis. Differences expected. 12] M --> P[Motor imagery explored to
control decoded movements. 13] G --> Q[Part 3: synergies in
technologies, applications. 14] Q --> R[HEXO: synergy-embedded 10DOF hand
exoskeleton. EMG control. 15] Q --> S[Synergy-based training hypothesized to
enable better task transference. 16] Q --> T[Affordable synergy-based arm exoskeletons
for stroke rehab. 17] Q --> U[VR to train hand
'alphabet' synergies. 18] Q --> V[Biomimetic learning: teach robots
via synergy approaches. 19] A --> W[Human-robot interaction: fusing expressions,
signals improves emotion recognition. 20] A --> X[Wearable EEG/EDA systems detect
stress, provide interventions. 21] A --> Y[Future research: model brain
for real-time assistive robotics. 22] A --> Z[NSF center BRAIN launched
to accelerate neurotech. 23] A --> AA[Mentors, collaborators, students
acknowledged. 24] A --> AB[AI/ML, AR/VR, neurotech: key
themes in brain-robotics. 25] AB --> AC[Non-invasive BCI potential as
computing power increases. 26] AC --> AD[Deep learning to handle
real-world BCI noise. 27] A --> AE[Spring school: 68,900 people/day
via live streams. 28] A --> AF[Q&A: opportunities, advances, ethics,
challenges. 29] A --> AG[Impressive research. Further discussions,
collaborations anticipated. 30] class A,B,G,H,I,J,K,L,M,N,O,P,Q synergy; class R,S,T,U,V,W,X applications; class C,D,E,F,Y,Z,AB,AC,AD research; class AA,AE,AF,AG acknowledgments; class future future;

Resume:

1.- Presenter Ramana Kumar Vinjamuri discusses synergy-based brain-machine interfaces from a human-robot interaction perspective.

2.- Goal is understanding how the intelligent human brain controls the dexterous human hand and processes multi-input, multi-output sensory information.

3.- Two approaches: top-down understanding neural representations of hand movements and bottom-up understanding control mechanisms of hand movements.

4.- Applications are improving impaired motor control, augmenting motor control, improving motor rehabilitation post-stroke, and accelerating new motor skill learning.

5.- Central topic is synergies - building blocks/alphabet of movement shared across movements. Combining synergies enables performing many movements.

6.- First part focuses on dimensionality reduction in control of hand movements, going from behavior to brain.

7.- Synergies are extracted from hand kinematics using dimensionality reduction methods like PCA. Weights optimally combine synergies to reconstruct movements.

8.- Linear methods efficiently capture global movement patterns. Data fusion of kinematics and muscle activity improves synergy representation and movement reconstruction.

9.- Dynamical synergies from contact forces and pressures inform about population activation and pressure points during grasps.

10.- Second part is on neural representations of synergy-based hand movements using non-invasive methods, going from brain to behavior.

11.- Simplified synergy combination model used with EEG to decode synergy-based hand movements. Spectral EEG features enable 90%+ movement reconstruction accuracy.

12.- Work being extended to individuals with paralysis. Differences in EEG expected compared to healthy subjects.

13.- Motor imagery being explored to control movements decoded from models trained on motor execution EEG features. Encouraging preliminary results.

14.- Third part covers applications of synergies in technologies like exoskeletons, assistive devices, humanoid robot interaction, and motor learning.

15.- HEXO is a synergy-embedded 10DOF hand exoskeleton with low-dimensional control of a high-dimensional system. EMG sensors control functional synergies.

16.- Synergy-based training hypothesized to enable better transference to new tasks compared to task-based training. Preliminary results support this.

17.- Synergy-based arm exoskeletons being developed in India for affordable stroke rehab. Moving beyond 2D to 3D control using arm synergies.

18.- Immersive VR environment being created to train people to learn hand movement "alphabets" or synergies to achieve more dexterous control.

19.- Biomimetic learning aims to teach robots new movements through synergy-based approaches rather than only imitation learning for expanded capabilities.

20.- Emotion recognition for human-robot interaction being improved by fusing facial expressions with neurophysiological signals like EEG for higher accuracy.

21.- Wearable integrated systems using EEG/EDA being developed to detect stress, a first symptom in many mental health disorders, and provide interventions.

22.- Future research includes modeling basal ganglia and cerebellum to find synergy representations for real-time control of assistive robotics and transference in motor learning.

23.- NSF industry-university cooperative research center BRAIN launched at UMBC to work with neurotech companies to accelerate technologies to market.

24.- Mentors, collaborators and especially students across many institutions acknowledged for enabling the impactful research.

25.- Tremendous growth expected at intersection of brain and robotics in next 5-10 years. AI/ML, AR/VR, neurotech identified as key themes.

26.- Transformative potential of non-invasive BCI anticipated as computing power increases. But important ethical concerns exist around privacy, autonomy, equity, etc.

27.- Studies so far in lab settings, but deep learning expected to help handle real-world noise in non-invasive BCI deployed in unconstrained environments.

28.- Spring school reached an astounding average of 68,900 people per day across multiple simultaneous live streams around the world.

29.- Engaging Q&A followed the presentation touching on opportunities at UMBC, collaborations, advances in the field, transference, ethical issues, and practical challenges.

30.- Highly impressive and impactful research and an excellent presentation much appreciated by the audience. Further discussions and collaborations anticipated.

Knowledge Vault built byDavid Vivancos 2024