Subjective Tests for Quality of Experience in Streamed Virtual Reality Games

Abstract: VR gaming is an emerging market with relatively new technology when compared to PC and console gaming. Streamed VR gaming is when the VR game is being streamed from another device to the VR headset. This experiment will show how the VR hardware and software performs under network constraints such as RTT and packet loss for streamed VR gaming specifically. User QoE (quality of experience) will be gathered and compiled into MOS (mean opinion score) for later use in graphs to determine negative trends. All collected data will also be sent to the Infovista technical team so they can make theoretical models on how user QoE is affected by network conditions. This thesis will show how the QoS (quality of service) factors, such as latency and packet loss, impact the users overall QoE and the VR headsets FPS. Before the subjective experiment could begin a testbed was chosen and setup, a VR game was decided on, and a streaming service was tested on. During the experiment the game Serious Sam VR: The Last Hope was played and NVIDIA’s CloudXR streaming service was used. The testbed setup consisted of two PCs, one router, one access point, a tablet used for user reviews, and a VR headset with two handheld controllers. The software program ALTRUIST was used for managing the user data and applying the desired network conditions such as latency. Each user experiment tested 28 different network conditions with varying values of RTT (round trip time), Packet loss, RTT with packet loss, and RTT with random jitter. 30 users were gathered for the actual experiment where the Calendly application was used to manage the user appointments. The results consisted of objective and subjective data gathered during the experiments. From the objective data it could be determined that RTT primarily affects the FPS in the VR headset. Packet loss affected the FPS but to a minor degree compared to RTT. RTT of 175 ms received an average FPS of 53 while 24% packet loss yielded approximately 65 FPS. Random jitter had close to no impact on the FPS according to the results. The subjective data suggests that RTT of 75 ms and below yields a good user MOS and RTT of 175 ms gives an acceptable user MOS at 3 out of 5. Packet loss of 6% and below gives a good user MOS while 12% and above results in poor MOS. RTT with random jitter performs average to poorly with Standard deviation of 3 and above, for low RTT values. For future experiments it is advisableto use ALTRUIST or a similar software to help the researcher during the experiment.