Detecting Offside Position using 3D Reconstruction

Abstract: With the increased number of technical innovations in soccer and the progress in the field of AI, this thesis investigates the possibilities to detect if a player is in offside position using deep learning and video recordings from soccer games. A method to create a 3D reconstruction of the players in the scene is proposed and evaluated. The arguments for using a 3D reconstruction is that it introduces a volume to the players which hopefully result in increased quality of offside decisions. A system for detecting offside position is proposed consisting of an object detector, a pose estimator, a mesh estimator and a team classification. The position of the players are estimated by minimizing the reprojection error between the 2D pose and the 3D pose, further on the reprojection is weighted using the confidence of each predicted 2D joint. The system is able to detect players in offside position with a precision of 60% and a recall of 97%. It is also shown that by weighting the reprojection error the precision of detected players in offside position is increased from 5\% to 60%. It is also shown that reconstructing the scene increases the average F1 score from 0.82 to 0.86.