Short Range Gesture Sensing and Classification Using Pulsed Millimeter-Wave Radar and Convolutional Neural Networks

Abstract: Human hand gestures present a novel way of interacting with electronic devices. A millimeter-wave radar setup utilizing a pulsed Resonant-Tunneling Diode signal generator in the 60 GHz ISM band is used to measure 12 different hand gestures. The data is used to train and validate Convolutional Neural Networks (CNNs). The measurement setup utilizes a real time sampling oscilloscope and down-mixing of the received radar signal. Three data types are under test: data without processing (Range-Time), Fourier-transformed data (Range-Doppler) and windowed Fourier-transformed data divided into three frames (WFT). A pre-defined and pre-trained CNN ResNet50 is initially used for data classification. The validation accuracy for 12 gestures with 180 measurements each and a training-validation quota of 60%-40% was 98% (Range-Time), 85% (Range-Doppler) and 91% (WFT). Additionally, a proposed CNN architecture with less complexity named SimpleNet is investigated, showing a validation accuracy (for the same data and training-validation quota as for ResNet50) of 95% (Range-Time), 85% (Range-Doppler) and 93% (WFT). For SimpleNet, the presented results are the average values of 25 different training sessions. Additionally, measurements from an independent test group were classified using above trained networks, with results that indicated relatively weak generalization for the classifying networks under test.