Knowledge Vault 3/39 - G.TEC BCI & Neurotechnology Spring School 2024 - Day 4
Neurogames 101: state-of-art, guidelines and challenges to
build an accessible and enjoyable BCI experience
Michele Romani, g.tec medical engineering GmbH (AT)
<Resume Image >

Concept Graph & Resume using Claude 3 Opus | Chat GPT4 | Llama 3:

graph LR classDef phd fill:#f9d4d4, font-weight:bold, font-size:14px; classDef neurogames fill:#d4f9d4, font-weight:bold, font-size:14px; classDef bci fill:#d4d4f9, font-weight:bold, font-size:14px; classDef games fill:#f9f9d4, font-weight:bold, font-size:14px; classDef tips fill:#f9d4f9, font-weight:bold, font-size:14px; A[Michele Romani] --> B[PhD student, BCI in gaming. 1] A --> C[Neuro game: brain-controlled mechanics. 2] A --> D[Games drove computer, gaming industry. 3] A --> E[First brain game: 1970s. 4] E --> F[BCI games: training, treatment,
rehabilitation, accessibility. 4] A --> G[BCI applications: clinical, non-clinical. 5] A --> H[BCI components: user, processing,
extraction, training, interface. 6] A --> I[Visually evoked potentials used. 7] A --> J[Michele's Unity BCI interface. 8] J --> K[Brain Hockey, Green Shield developed. 9] K --> L[Training phase calibrates classifier. 10] L --> M[Training collects data, trains,
tests classifier. 11] K --> N[In-game metrics: accuracy, transfer rate. 12] K --> O[Pilot testing: 80%+ accuracy in
Green Shield, improvement on repeats. 13] K --> P[Brain Hockey harder, 60% accuracy
beats AI, scores improved. 14] K --> Q[100% accuracy over 90 enemies
in Green Shield. 15] K --> R[Classifier probabilities validated results. 16] A --> S[Pitfalls: signal quality, calibration,
confidence, visual design. 17] S --> T[Poor signal makes game unreliable. 18] S --> U[30+ trials for calibration,
60 ideal. 19] S --> V[Confidence balances precision vs speed. 20] S --> W[Avoid transparent, overlapping,
moving stimuli. 21] A --> X[Tips: understand BCI, playtest,
choose paradigm. 22] X --> Y[Gamify calibration, use hybrid controls. 23] X --> Z[Vary focus time for difficulty. 24] A --> AA[Challenges: reduce calibration,
improve reliability, reduce waste. 25] A --> AB[On-device classification, artifact removal. 26] A --> AC[New subject-specific models each session. 27] A --> AD[Exploring persistent models,
pre-training, auto-training. 28] A --> AE[Fatigue an issue, breaks help,
motor imagery an option. 29] A --> AF[Minimal rear-head setup suffices. 30] class A,B phd; class C,D,E,F,G neurogames; class H,I,J,K,L,M,N,O,P,Q,R,AA,AB,AC,AD,AE,AF bci; class S,T,U,V,W,X,Y,Z tips;

Resume:

1.- Michele is a PhD student working on bringing brain-computer interface (BCI) technology into gaming.

2.- He defines a neuro game as one whose mechanics are partly or entirely controlled by brain input.

3.- Games have been a driving factor in the computer and gaming industry over the past 50 years.

4.- The first brain game dates back to the 1970s. BCI games have been used for training, treatment, rehabilitation, accessibility.

5.- Main applications are clinical (accessibility, rehabilitation) and non-clinical (serious gaming for research, entertainment).

6.- A BCI system has components of user, data processing, feature extraction, classifier training, and a control interface (the game).

7.- The games discussed use visually evoked potentials - single stimulus, steady-state (SSVEP), and code modulated.

8.- Michele developed a Unity interface to easily create BCI games, handling signal processing and classification.

9.- Brain Hockey (Pong-like) and Green Shield (Space Invaders-like) are two games he developed using the single stimulus method.

10.- Games have a training phase to calibrate the classifier to the player's brain signals. UI elements indicate good or bad calibration.

11.- Under the hood, the training collects brain data for the target stimuli, trains a classifier, tested with cross-validation.

12.- In-game, performance metrics are task accuracy (using the right input for the game goal) and information transfer rate.

13.- In pilot testing, players achieved 80%+ accuracy on their first try in Green Shield. Scores improved on repeat sessions.

14.- Brain Hockey felt harder to players. 60% accuracy can beat the AI. Scores also improved on additional tries.

15.- To show reliability, Michele scored 100% accuracy on 90 consecutive enemies over 3 rounds in Green Shield.

16.- Classifier probability plots matched the expected target sequence, validating the classification results.

17.- Common pitfalls in neuro games include poor signal quality, too short calibration, non-ideal confidence levels, suboptimal visual design.

18.- Signal quality can be checked pre-game. Poor quality makes the game unreliable.

19.- 30 trials minimum recommended for calibration, 60 is ideal. Confidence levels balance precision vs speed.

20.- High confidence means high precision but slower. Low confidence is faster but less precise. Design depends on number of targets.

21.- Transparent stimuli and overlapping/moving targets make classification harder and should be avoided.

22.- Design tips: understand BCI constraints, playtest yourself and with others, design around game affordances, choose the right BCI paradigm.

23.- Gamify the calibration phase to make it engaging. Can use both BCI and standard inputs in hybrid control schemes.

24.- Vary focus time required for classification to adjust difficulty level.

25.- Current challenges: reducing calibration time through persistent models and pre-training, improving reliability with fewer channels, reducing data waste.

26.- Classification and artifact removal are done on-device. Uses a modified linear discriminant analysis classifier.

27.- Each game session currently trains a new subject-specific model. Preserving data across sessions being explored.

28.- Pre-trained multi-subject models and auto-training during gameplay are avenues to reduce calibration time.

29.- Fatigue is an issue, especially with flashing stimuli. Breaks help. Alternative BCI paradigms like motor imagery are options.

30.- Only occipital channels over the visual cortex are needed for visually evoked BCI games. A minimal rear-head setup suffices.

Knowledge Vault built byDavid Vivancos 2024