Vr-Eeg System for Gamified Assessment

Abstract

Assessing the cognitive workload is a fundamental need for the design of intelligent and adaptive human computer interaction (HCI) systems, especially in immersed environments where the difficulty level of a task can dynamically affect a user’s mental state. In this research, a VR-EEG framework for cognitive workload assessment based on the electroencephalography (EEG) signal during a memory-based VR task is introduced, which is gamified to make the task more realistic. The EEG data were recorded from 16 participants using an OpenBCI Cyton 8-channel biosensing board when participants were asked to complete tasks under four different workload conditions: Relaxation, Easy, Medium, and Hard. Three different deep learning models were designed and tested under the same experimental setup. The first model, MultiBranch Attentive EEGNet (MBA-EEGNet), uses multi-branch temporal feature extraction, depthwise spatial convolution and channel attention to efficiently learn discriminative EEG representations. The second model is Multi-Scale Dynamic Graph Cross-Attention Conformer (MS-DGCA Conformer), which combines multi-scale convolution, dynamic graph-based spatial learning, cross-attention, and Conformerbased feature extraction to model the spatial and temporal relationships of EEG signals. Last, the proposed MS-DGCA-MBA-EEGNet Hybrid model is an integration of complementary features of MBA-EEGNet and MS-DGCA Conformer that aims to enhance the representation of both temporal EEG dynamics and spatial connectivity patterns. The evaluation with the topics proved the mean classification accuracy of MBA-EEGNet as 79.23% and that of MS-DGCA Conformer as 76.74% when subject wise evaluation was performed. The MS-DGCA Conformer also showed a good mean accuracy of 78.28% across 5 folds of the cross validation, which shows that it is generalizing well. The proposed MS-DGCA-MBA-EEGNet Hybrid model performed best on all the subjects (81.50% mean, across the 16 participants) compared to the individual architectures. The findings in this work show the power of this new approach that integrates convolutional, attention-based, graph-based, and Conformer-based feature learning to provide accurate multi-level cognitive workload estimation in immersive VR applications. A cognitive workload assessment of VR users in the past required the researcher to observe from afar, carry VR equipment, or choose the appropriate participant. Keywords: EEG, Cognitive Workload Assessment, Virtual Reality, MBA-EEGNet, MS-DGCA Conformer, Hybrid Deep Learning, Brain–Computer Interface, EEG Classification, Multi-Scale Feature Learning