The End Of Knowledge - Vault 3 - G.TEC BCI & Neurotechnology Spring School 2024 - Day 3 - Ethel Pruss, Caterina Ceccato, Anita Vrins, Jos Prinsen

graph LR classDef team fill:#f9d4d4, font-weight:bold, font-size:14px; classDef internship fill:#d4f9d4, font-weight:bold, font-size:14px; classDef neurowizard fill:#d4d4f9, font-weight:bold, font-size:14px; classDef multiplayer fill:#f9f9d4, font-weight:bold, font-size:14px; classDef darkReflection fill:#f9d4f9, font-weight:bold, font-size:14px; A[Brainy Hacks] --> B[BCI experts,
hackathon winners. 1] A --> C[Internship: unicorn BCI
game development. 2] C --> D[Anticipating internship game reveal. 3] B --> E[Katerina, Anita, Jos, Ethel, Mariam.
Cognitive Science, AI students. 4] B --> F[3 years of Brain.io hackathons,
successful projects. 5] B --> G[Thanks GTEC for
hosting, Austria visit. 6] A --> H[BCI games: research,
medical applications. 7] H --> I['Neurowizard': gamified BCI
paradigm testing. 8] I --> J[Fun, engaging BCI
selection vs standard. 9] I --> K[Text-based RPG: dungeon,
spell casting. 10] I --> L[Minimal, understandable
first-time user interface. 11] I --> M[Gamification: motivation, immersion,
long-term replay. 12] I --> N[Design: challenge, theme,
reward, progress, art. 13] I --> O[Calibration integrated naturally,
simple control explanation. 14] I --> P[Visual orb color
classifications, avoid percentages. 15] I --> Q[Adaptive difficulty: increasingly
complex spell casting. 16] I --> R[Prototype: gamifying beyond
simple BCI tasks. 17] H --> S[Medical BCI games
e.g. stroke recovery. 18] S --> T[Multiplayer game: motivating,
social motor training. 19] T --> U[Paradigm combo: P300, SSVEP.
Turn-based, ability-inclusive. 20] T --> V[4 characters, BCI paradigms:
P300, SSVEP, motor, passive. 21] T --> W[Collaborative or competitive.
Monster battles, challenges. 22] T --> X[24-hour hackathon
playable Unity demo. 23] H --> Y[Internship project: immersive
full-scale BCI game. 24] Y --> Z[BCI game challenges:
classifier speed, plug-and-play. 25] Y --> AA[Research/medical vs
indie/AAA BCI games. 26] Y --> AB['Dark Reflection': story-based
BCI mystery solving. 27] AB --> AC[BCI enhances gameplay,
blends with story. 28] Y --> AD[Rich potential: engaging
worlds, intuitive BCI. 29] class A,B,C,D,E,F,G team; class H,Y,Z,AA,AB,AC,AD darkReflection; class I,J,K,L,M,N,O,P,Q,R neurowizard; class S,T,U,V,W,X multiplayer;

Resume:

1.-Christoph introduces the Brainy Hacks team, who have done nice work in the BCI domain and won past BCI Hackathons.

2.-The Brainy Hacks team spent half a year doing an internship on game development with GTEC's new unicorn BCI interface.

3.-Christoph is excited to see the outcome of the game the team developed during their internship over the past half year.

4.-The team introduces themselves - Katerina, Anita, Jos, Ethel and Mariam. They are master's students in Cognitive Science and AI.

5.-The team has participated in Brain.io hackathons for about 3 years and made successful projects, working together on interesting small projects.

6.-The team gives a shout-out to GTEC for hosting them as interns. They visited the headquarters in Austria.

7.-BCI games can be used in research contexts and in the medical field, which Joss will explain further.

8.-Anita presents her research project "Neurowizard" - a gamified BCI for paradigm testing using a wizard character casting spells.

9.-The game aims to make the BCI selection task more fun and gamified compared to a standard minimalistic application.

10.-The game uses text-based RPG-style with the user going through dungeon rooms and casting spells by selecting items in order.

11.-As a first-time user, the game interface is designed to be minimal and understandable. It was developed in Unity.

12.-Gamification can provide motivation for long-term use and replaying. Users may prefer an immersive game vs a flat interface.

13.-Design process considered gamification literature - incorporating challenge, theme, reward, progress. Used AI art, own drawings, animations.

14.-Anita integrated the calibration into the game narrative naturally. Explains controls simply without BCI jargon for new users.

15.-Classifications represented visually as orb colors to avoid confusing new users with accuracy percentages. Translates to game progression.

16.-Game adapts difficulty with increasingly complex selection tasks framed as casting more advanced spells. Motivates improvement.

17.-Prototype shows potential of gamifying BCI apps beyond simple tasks. Engaging narrative and visuals enable longer, immersive experiences.

18.-Next, Ethel presents their attempts at medical field BCI games, e.g. for stroke recovery which can be tedious.

19.-During last year's hackathon, they made a multiplayer game to motivate patients and enable social connection during motor imagery training.

20.-They combined paradigms (P300 and SSVEP) to connect users with different abilities. Turn-based to allow different timing windows.

21.-Four characters each with own attack style and matching BCI paradigm - P300, SSVEP, motor imagery, passive (valence-based).

22.-2-player collaborative or competitive game. Defeat monsters together or challenge each other. Motivating way to support motor-impaired users.

23.-Developed basic playable demo in Unity within the 24-hour hackathon. Shows potential for making BCI games quickly.

24.-Jos overviews their internship project developing a near full-scale immersive BCI game in Unity as a proof-of-concept.

25.-Contrasts research/medical focused BCI games with limited scope vs full-fledged immersive games that appeal to a broad audience.

26.-Challenges in making BCI games for the public - speed/information transfer rate of classifiers, lack of plug-and-play devices.

27.-Large gap between research/medical BCI apps vs indie/AAA video games in mechanics, scope, visual design, immersion.

28.-Their game "Dark Reflection" is story-based to allow natural onboarding. Users solve a mystery using BCI "mind-reading" mechanics.

29.-BCI interactions enhance gameplay, not replace controls. Blend naturally into narrative, environment, conversations.

30.-In-progress BCI game shows potential for rich experiences combining engaging worlds and stories with intuitive BCI enhancements.

Knowledge Vault built byDavid Vivancos 2024