Development of a full-body interaction videogame for children with Autism based on generative graphics and a Kinect-based tracking system

Abstract

The main objective of this research project is to develop the tracking and virtual system framework to support a multiplayer game specially destined for children with Autism. With this system, the players are expected to interact and communicate with each other in order to make them develop new social and communication behaviors. To do so, this project is based on a full body interaction system. In order to create such a system, the game uses a Kinect-based tracking system, allowing us to detect gestures as well. Concretely, two Kinects are being used and connected via an OSC connection. The game that will be developed for this platform will be implemented using Unity, a game engine program. Currently, a prototype application has been developed also in Unity to be able to test the technology developed in this TFG. With the objective of bringing this system to special schools and hospitals, the set-up will also be portable. This way it will be easily transportable, and adaptable to different classrooms sizes and configurations.El principal objectiu d’aquest projecte és desenvolupar un sistema virtual i de seguiment que suporti un joc multi jugador, especialment dissenyat per a nens amb Autisme. Amb aquest sistema, els jugadors podran interactuar i comunicar-se entre ells per tal que puguin desenvolupar nous comportaments tant socials com comunicatius. Per això, aquest projecte es basa en un sistema d’interacció de cos sencer. Per tal de crear el sistema el joc utilitza el sistema de seguiment de la Kinect, permetent-nos la detecció de gestos de l’usuari. Concretament, s’utilitzen dues Kinects i estan connectades via OSC. El joc ha estat implementat amb Unity. Actualment, s’ha desenvolupat, també en Unity, un prototip per tal de testejar la tecnologia utilitzada en aquest TFG. Amb l’objectiu de portar aquest sistema a les aules i hospitals, aquest serà portable per tal que sigui fàcil de transportar, i s’espera fer-ne diferents versions pel que fa a les seves dimensions per tal d’adaptar-se a diferents classes i configuracions.