Intuitive Human Robot Interaction and Workspace Surveillance by means of the Kinect Sensor

Abstract: This student thesis is part of the ROSETTA project that examines the possibilities of future manufacturing based on close human-robot interaction. In this report, the capabilities of a robot system that is controlled by the Kinect sensor of the Microsoft XBOX-360 are tested. A setup including two Kinect devices and one robot is presented. Applications for calibration of two Kinect sensors, robot workspace surveillance, face detection and shadowing the user's arm are introduced and evaluated concerning their effectiveness. Finally, the compatibility of the introduced setup with the objectives of the ROSETTA project is evaluated. It proves that the Kinect can strongly support developing solutions for these objectives. The thesis work was conducted at the Department of Automatic Control at LTH at Lund University, Sweden, in collaboration with the Institute of Integrated Sensor Systems at University of Kaiserslautern, Germany.