Knowledge Vault 3/35 - G.TEC BCI & Neurotechnology Spring School 2024 - Day 3
Immersion and how to have a realistic experience based on cognitive factors
Jonas Vibell, University of Hawaii (USA)
<Resume Image >

Concept Graph & Resume using Claude 3 Opus | Chat GPT4 | Llama 3:

graph LR classDef presenter fill:#f9d4d4, font-weight:bold, font-size:14px; classDef background fill:#d4f9d4, font-weight:bold, font-size:14px; classDef hawaii fill:#d4d4f9, font-weight:bold, font-size:14px; classDef vr fill:#f9f9d4, font-weight:bold, font-size:14px; classDef immersion fill:#f9d4f9, font-weight:bold, font-size:14px; classDef model fill:#d4f9f9, font-weight:bold, font-size:14px; classDef equipment fill:#f9d4d4, font-weight:bold, font-size:14px; A[Jonas Vibell] --> B[Univ Hawaii assistant professor 1] A --> C[Jonas' background:
Oxford, UCL, MGH 2] A --> D[Hawaii lab:
neuroimaging, rehab, VR 3] D --> E[VR unit:
EEG, rehab applications 4] A --> F[Immersion: surrounded
by different reality 5] F --> G[Immersion range:
books to VR 6] F --> H[Low-fidelity VR
induces strong immersion 7] F --> I[Narrative, logic
maintain immersion 8] F --> J[Immersion: sensory
input, imagination 9] F --> K[Early VR blocky,
modern realistic 10] F --> L[Advanced VR
mirrors real activities 11] F --> M[High immersion: multisensory
fidelity, congruence 12] M --> N[Multisensory brain integration
enhances immersion 13] F --> O[True VR:
many senses needed 14] F --> P[Internal immersion:
content over presentation 15] F --> Q[Top-down factors
influence immersion 16] F --> R[Attention studied,
immersion needs work 17] A --> S[Immersion model:
internal/external factors 18] S --> T[Immersion continuum:
2D to haptics 19] A --> U[VR for neurorehab:
effective therapies 20] A --> V[Preliminary model:
attention and immersion 21] A --> W[Physiological immersion measures
advance applications 22] A --> X[Lab equipment:
HTC Vive Elite 23] A --> Y[Cybersickness issue:
fast movements, headsets 24] F --> Z[Immersion and graphics
fidelity connected 25] A --> AA[Presenter seeks
collaboration from Hawaii 26] F --> AB[Quantifying context on
immersion needs definition 27] F --> AC[Immersion factors overlap,
need definitions 28] F --> AD[Over-immersion trauma
not well studied 29] F --> AE[Real-world immersion
a deep question 30] class B,C presenter; class D,E,U,X hawaii; class F,G,H,I,J,K,L,M,N,O,P,Q,R,Z,AB,AC,AD,AE immersion; class S,T,V,W model; class Y equipment;

Resume:

1.- The presenter, Jonas Vibell, is an assistant professor at the University of Hawaii, directing the Brain and Behavior Laboratory.

2.- Jonas' research background spans institutions like Oxford, UCL, MGH, and a company called ArchinoEthics that developed brain-related technologies.

3.- At the University of Hawaii, Jonas' lab has three units: neuroimaging, neurorehabilitation, and virtual reality research.

4.- The virtual reality unit combines VR with EEG to understand brain function and applies it to domains like rehabilitation.

5.- Immersion is a key concept in VR research - it's the feeling of being surrounded by a completely different reality.

6.- Immersion can range from reading an engrossing book with no sensory input to highly realistic multisensory VR experiences.

7.- Even low-fidelity VR can induce strong feelings of immersion, like fear of walking a virtual plank at a great height.

8.- Narrative and logic within the virtual world are important for maintaining the sense of immersion.

9.- Immersion depends on both bottom-up sensory input and top-down factors like imagination, spanning internal and external attention.

10.- Early VR had blocky graphics and clunky headsets that broke immersion, but modern VR achieves much greater realism.

11.- Advanced VR like Meta's can mirror many real-world activities, from exercise to social interaction to travel.

12.- High immersion requires fidelity and congruence between senses - visual, auditory, touch, smell - spatially, temporally, and semantically.

13.- Multisensory integration in the brain can enhance immersion, with activation greater than the sum of individual sensory inputs.

14.- Many senses beyond the traditional five, like balance and mechanoreception, may be needed for true VR immersion.

15.- Internal attention and imagination, as used when immersed in a book, rely more on content than presentation.

16.- Factors like attention, emotion, expectation, motivation, memory influence immersion through top-down internal cognitive processes.

17.- Studying attention for decades has yielded insights relevant to immersion, but much work remains to precisely define immersion.

18.- A model of immersion should integrate external and internal attention, motivation, task factors, and the mapping between virtual and real.

19.- Immersion exists on a continuum from 2D interfaces to full VR with advanced haptics, all integrating multiple cognitive factors.

20.- Immersion is applied in domains like neurorehabilitation, where integrating VR and neurophysiology could enable more effective therapies.

21.- A preliminary model links immersion to an interplay of cognitive factors, especially attention, though more work is needed.

22.- Establishing physiological measures of immersion would greatly advance VR applications in education, rehabilitation, entertainment, and more.

23.- The lab uses an HTC Vive Elite VR system, which is lightweight and comfortable compared to older models.

24.- Cybersickness remains an issue in VR for some users, especially with fast movements, and differs between headsets.

25.- Suspension of disbelief in VR depends on fidelity - blocky graphics break immersion while realistic graphics maintain it.

26.- The presenter would like more collaboration, being relatively isolated in Hawaii, and shared his contact info.

27.- Quantifying the effect of context on immersion requires first establishing a rigorous definition of immersion.

28.- Distinguishing between attention, presence, agency in immersion is tricky as the concepts overlap - careful definitions are key.

29.- Over-immersion, in the sense of panic or trauma from a VR experience, is an interesting consideration not yet well-studied.

30.- Whether we are immersed in the real world through attention/intention is a deep philosophical question, perhaps impossible to test.

Knowledge Vault built byDavid Vivancos 2024