Visual assessments of Postural Orientation Errors using ensembles of Deep Neural Networks

Abstract: Injuries to the Anterior Cruciate Ligament (ACL) are severe and common among the physically active young to middle aged population. After suffering from such an injury, the patient typically face a lengthy rehabilitation process. Usually, it takes 1-2 years before an injured knee returns to pre-injury performance, if that is ever achieved. The risk of re-injury is high and is increased by early return to sports. One measure which has been suggested as an indicator of the increased risk of re-injury, and hence could work as an indicator of when to return to normal activity, is altered postural orientation. The postural orientation describes the positions of different body parts in relation to each other and the surroundings. Assessment of this is a time consuming task requiring human experts trained to find such alterations. This thesis propose a method to automate this task by analyzing videos recorded with a regular video camera, e.g. a mobile phone. The proposed method uses well established deep learning techniques, in this case HRNet with DARK-pose, to extract body part positions from each video frame. Deep learning based models are trained in a supervised fashion to classify the sequences of extracted keypoints. Models trained to perform according to different metrics were combined in ensembles classifying the quality of the postural orientation on an ordinal scale from 0 (Good), via 1 (Fair), to 2 (Poor). We evaluated the method on four different segment-specific Postural Orientation Errors (POEs) when the patient performed a single leg squat. The different POEs were trunk, pelvis, femoral valgus, and Knee Medial-to-Foot Position (KMFP). For femoral valgus and trunk a classification accuracy of 82.3% and 80.0%, respectively, was achieved. The corresponding number for KMFP was 90.3%, but this data was heavily imbalanced. The pelvis was the most difficult to analyze resulting in an accuracy of 73.3%. The most important contribution of this thesis is to provide a foundation and a number of insights of what is needed before introducing a method like this for clinical use.