Foreground detection in specific outdoor scenes : A review of recognized techniques and proposed improvements for a real-time GPU-based implementation in C++

Abstract: Correct insertion of computer graphics into live-action broadcasts of outdoor sports requires precise knowledge of the foreground, i.e. players present in the scene. This thesis proposes a foreground detection and segmentation- framework with focus on real-time performance for 1080p resolution. A dataset consisting of four scenes; single-, multi-segment-, transcending-foreground and a light-witch scene all with dynamic backgrounds was constructed together with 26 ground-truths. Results show that the framework should run internally at 288p using GPU acceleration with geometrical nearest-neighbour-interpolation to attain real-time-capability. To maximize accuracy of the results, the framework uses two instances of OpenCV MOG2 in parallel on differently downsampled frames that are bitwise-joined to increase robustness. A set of morphological operations provides post-processing to get spatial coherence and a specific turf- consideration gives accurate contours. Thanks to additional camera- operator input, a crude distance-estimate lets foreground segments fade into background at a predetermined depth. The framework suffers from inaccurate segmentation during rapid light-switches, but recovers in a matter of seconds like the 'vanilla' MOG algorithm. For the specific scenes the framework provides excellent performance, especially considering the light-switch scene by comparison to the MOG-algorithm. For non-specific scenes of the 'BMC 2012' performance does not exceed the current state-of-the-art.