Kalman filters as an enhancement to object tracking using YOLOv7

Abstract: In this paper we study continuous tracking of airplanes using object detection models, namely YOLOv7, combined with a Kalman filter. The tracking should be able to be done in real-time. The idea of combining Kalman filters with an object detection model comes from the lack of time-dependent context in models such as YOLOv7. The model analyzes each frame independently and outputs airplane detections for the analyzed frame. Therefore, if an airplane flies behind a tree or a cloud, the object detection model will say that there is no object there. The Kalman filter is used to construct an object with a state consisting of position and velocity for every airplane. As such if an airplane flies behind a tree, it is possible to extrapolate the trajectory and resume tracking once the airplane is visible again, much like a human would extrapolate the trajectory naturally. In the report I describe the implementation and training of a YOLOv7 model, I further describe the construction and implementation of a Kalman filter as well as how observations are mapped on to objects in the Kalman filter. During this I introduce a parameter called cumulative confidence. This describes how long something is being tracked after observations cease. After losing sight of an object, the cumulative confidence starts to drop. When it reaches zero and the object is removed. This can take anywhere between 100 ms to 6 seconds depending on how much confidence the object has accumulated. Objects accumulate confidence by being observed and detected by the object detection model. In the results section I describe how the performance of the program changed when using a Kalman filter or when not using a Kalman filter. The results showed that continuous tracking of airborne airplanes was superior when using a Kalman filter as opposed to only using the YOLOv7 model. Continuous tracking was never lost in these 2 airborne cases when using the integrated Kalman filter. Continuous tracking was lost 5 respectively 11 times on the same cases when not using the Kalman filter. The last case in the results section, an airplane on a runway, showed the same performance with and without the Kalman filter. I go into detail why this is in both the results section and in Section 5.1 (Interpreting the results).