Wielomodalna architektura głębokich sieci neuronowych do wczesnego przewidywania napadów epilepsji z wykorzystaniem EEG, sygnałów z urządzeń noszonych i wzorców behawioralnych

Abstract Epilepsy is a disorder that affects millions of people in the world, and proper early prediction of seizures has been a very critical issue in clinical neurology because the dynamics of the brain are complex and heterogeneous. The current methods are mainly based on a single-modality signal like EEG and thus have limited capacity to image both physiological and behavioral patterns, and many of them are weak in terms of fusion strategies and patient specific adjustability. To overcome these weaknesses, this paper presents a new multimodal deep learning architecture, which incorporates EEG data, wearable physiological data, and behavioral patterns with a reliability-conscious cross-modal attention system. The framework uses a hybrid model that uses both temporal convolution and transformer-based encoders and adaptive personalization layers to improve the predictive performance. The multi-task learning approach is used to do both the seizure classification and time-to-seizure prediction at the same time. It is experimentally proven that the proposed model can be applied with the accuracy of 96.8, sensitivity of 95.4 and ROC-AUC of 0.97, which is much higher than the baseline unimodal and conventional fusion models. Moreover, the system is able to forecast the occurrence of a seizure up to 20 minutes before it occurs with a lower incidence of false alarms. The main contributions are a new combination strategy, the implementation of multimodal timing features, and patient-adaptive learning, which makes the framework much more appropriate to the real-world clinical application and to the further development of the state-of- the-art in seizure prediction systems.