Recording Rendering API Commands for Instant Replay : A Runtime Overhead Comparison to Real-Time Video Encoding

Abstract: Background. Instant replay allows an application to highlight events without exporting a video of the whole session. Hardware-accelerated video encoding allows replay footage to be encoded in real-time with less to no impact on the runtime performance of the actual simulation in the application. Hardware-accelerated video encoding, however, is not supported on all devices such as low-tier mobile devices, nor all platforms like web browsers. When hardware-acceleration is not supported, the replay has to be encoded using a software-implemented encoder instead. Objectives. To evaluate if recording rendering API commands is a suitable replacement of real-time encoding when hardware-accelerated video encoding is not supported. Method. An experimental research method is used to make quantitative measurements of the proposed approach, Reincore, and a real-time encoder. The measured metrics is frame time and memory consumption. The Godot game engine is modified with modules for real-time video encoding (H.264, H.265 and VP9 codecs) and rendering API command recording and replaying. The engine is also used to create test scenes to evaluate if object count, image motion, object loading/unloading, replay video resolution and replay video duration has any impact on the runtime overhead of frame time and memory consumption. Results. The implemented rendering API command replayer, Reincore, appears to have minimal to no impact on the frame time overhead in all scenarios, except for a spike in increased frame time when the replayer initializes synchronization. Reincore show to be overall inferior to real-time video encoding in terms of runtime memory overhead. Conclusions. Overall, real-time encoding using the H.264 or H.265 show a similar result in frame time as recording rendering commands. However, command recording implies a more significant overhead of memory usage than real-time encoding. The frame time of using the VP9 codec for real-time encoding is inferior to recording rendering API commands.