Speeding up a mixed reality application: A study of two encoding algorithms

Abstract: Mixed reality (MR) or augmented reality (AR) is a way of combining the digital world with the real world. It is utilized in a number of different applications. The mobile devices of today may not always provide sufficient processing power to handle these kind of applications. One way of increasing the amount of available processing power is to offload the processing to a computer that sits on the edge of the network, for example residing in part of an access network (5G or dedicated unit connected to a router today). The upside of doing this is that it allows you to have a lot more processing power using a more powerful computer instead of the mobile device itself. The downside is that it increases latency which can deteriorate the user experience. To be able to send video to and from the edge computer, the video frames have to be encoded using an encoding algorithm. In this study we implemented two different encoding algorithms, VP8 and H.265, into an existing open source mixed reality application and studied how they affect the responsiveness of the application compared to the already implemented encoders, as well as how they perform when encoding at different bitrates. It is shown that the encoding algorithm affects the responsiveness to a certain degree, where the VP8 codec was the overall best performer in terms of responsiveness combined with visual quality. One major reason is that the VP8 codec had much faster decoding compared to the other codecs. The bitrate affects the encoding and decoding speed where higher bitrates resulted in higher encoding times. Higher bitrates also led to the application becoming less responsive which show a correlation between the bitrate magnitude and the responsiveness.