Deep Learning for Driver Sleepiness Classification using Bioelectrical Signals and Karolinska Sleepiness Scale

Abstract: Driver sleepiness contributes to a large amount of all road traffic crashes. Developing an objective measurement of driver sleepiness in order to prevent eventual traffic accidents is desirable. The aim of this master thesis was to investigate if deep learning can be used to provide a driver sleepiness classification from brain activity signals obtained by electroencephalography (EEG). The intention was to study the classification performance when using different representations of the input data and to examine how various deep neural network architectures and class weighting during training affect the classification.  The data was collected from 12 experiments, where 269 participants (1187 driving sessions) were driving either on real roads or in a moving-base driving simulator, while electrophysiological data was recorded. Several deep neural network architectures were developed, depending on the representation of the input data.  Regardless of which data representation that was used as input to the network, the datawas divided into three datasets: Training 60%, validation 20% and test 20%. The data from each participant, with associated driving sessions, were randomly assigned to the different datasets according to the given percentage, which resulted in a subject-independent sleepiness detection. The output was in the form of continuous regression further rounded to the closest integer and divided into five classes according to Karolinska Sleepiness Scale (KSS = 1-5, 6, 7, 8, 9). The best performance was obtained with a convolutional neural network (CNN) combined with Long Short-Term Memory (LSTM) architecture, with time series data as input. This gave an accuracy of 41.44%, a mean absolute error of 0.94 and a macro F1-score of 0.37. Overall, the models with time series data showed better classification results compared to those with time-frequency data. Class weighting, giving all classes inverse proportional weight to their appearance, compensated slightly for class imbalance, but all networks had in general difficulties with generalizing to new data.