On the communication problems of virtual reality: responsiveness, timeliness, and effectiveness

Abstract

In this article, we explore communication problems in virtual reality (VR) systems from the perspective of wireless networking. Our starting point is that the dynamics of real-time interaction between the physical and virtual worlds cannot be fully captured by the frame rate. Particularly, the frame rate can be very high, but the actual user experience can be disappointing. We argue that the key idea is to think of VR as the mediator of the senses, especially vision, rather than a conventional video streaming application. This is indeed a remarkable opportunity to recall Weaver's classification and define three fundamental problems in VR systems: i) responsiveness, ii) timeliness, and iii) effectiveness. Responsiveness refers to the delay between an action in the physical world and its display in the virtual world. Timeliness is related to the frequency of updates imposed by the frame rate. Effectiveness considers the visual content of a video frame and its likelihood of extrapolation. Our framework offers a concise and objective description of VR systems, that allows us to evaluate performance, identify potential issues, and design impactful solutions.