Human Activity Recognition using Deep Learning and Sensor Fusion

Abstract: The subject of this thesis is human activity recognition in health care. The data set used is the SBHAR data set. It contains accelerometer and gyroscope of 30 different individuals walking, walking upwards, walking downwards, sitting, standing and lying. The research questions is: How can we, using deep learning, understand more about and potentiallly improve Human activity recognition or HAR for assisted ambient living? To answer the question we take aim at three things. The first being a comparison of different deep learning methods and the SVM, to the best classifier in a review article from fall 2017. The results are that all the used supervised methods have> 90% accuracy in ten fold cross validation and display accuracies on par with the best classification result in the review article. Unsupervised methods all gave > 90% results for a 3 class problem. Secondly we aim for possible cost reductions. What neural networks use to distinguish between the different human activity recognition classes might be close to some handcrafted feature, which can replace a larger neural network. To find such patterns we visualized activations in the code of a convolutional autoencoder trying to reconstruct its input for the different walking classes. The latent space representations images were not easily distinguishable from each other. But, further reduction of the code reveal that the walking classes are mapped to two discs and a torus. Another possible cost reduction is replacing the less battery efficient gyroscope. With a regression network, We were able to reconstruct the gyroscope from the accelerometer with some likeness but without the faster fluctuations typically seen in gyroscope signals. Thirdly we evaluate two novel approaches, the MLP-HMM and SVM-MLP. The MLP-HMM and SVM-MLP are hybrids having one classier generating classifications and then a add on trying to model its errors by incorporating some previous and some future classifications. The SVM MLP hybrid is a combination of a support vector machine generating guesses and a multi layer perceptron modelling the guesses. The MLP-HMM hybrid uses a multi layer perceptron and a hidden markov model. The SVM-MLP hybrid shows a higher accuracy and lower standard deviation in comparison to only using the SVM. But the MLP-HMM hybrid gave worse results than if only using the MLP classier. The results are then followed by a discussion, were several suggestions for future work can be found regarding applications, sparser models, improvements and new classifiers.