SEMarbeta: Mobile Sketch-Gesture-Video Remote Support for Car Drivers

Abstract: Uneven knowledge distribution is often an issue in remote support systems, and this sometimes creates the need for additional information layers extending beyond plain videoconference and shared workspaces. This paper introduces SEMarbeta, a remote support system designed for car drivers in need of help from an office-bound professional expert. We introduce a design concept and its technical implementation using low-cost hardware and augmented reality techniques. In this setup, the driver uses a portable Android tablet PC while the helper uses a stationary computer equipped with a xxx-mounted video camera capturing his gestures. Hence, oral instructions can be combined with supportive sketches and gestures added by the helper to the car-side video screenshot. To validate this concept we carried out a user-study involving two typical automotive repair tasks: checking engine oil and examining fuses. Based on these tasks and following a between-group (drivers and helpers) design, we compared voiceonly with additional sketch- and gesture-overlay on video screenshot measuring objective and perceived quality of help. Results indicate that sketch- and gesture-overlay can benefit remote car support in typical breakdown situations.