Kinect Technology for Remote Assessment of Interventions for Young Children with Autism Spectrum Disorders

This project aims to provide innovative speech pathology services for children with Autism Spectrum Disorder (ASD) and their families in rural areas. The Kinect sensor plays a key role, being used as a novel remote feedback and assessment tool for the quality of parent-child interactions.

Kinect sensor data was collected at the Clear Messages speech pathology clinic in the Victorian regional town of Ballarat where volunteer parent-child sessions were recorded. This data was combined with further parent-child interactions recorded in March 2014 in the IBES Lab with 'neuro-typical' children, i.e. those unaffected by ASD, to act as a baseline against which ASD behaviours could be compared and contrasted. A variety of activities were recorded, including active and passive play such as jumping and dancing, shared book reading, and playing the 'Simon Says' mimicking game.

This project is developing an automated tracking and analysis system. The software provides meaningful statistics based on the quality of the parent-child interaction with a prototype dashboard display developed that takes the output of the Kinect sensor and displays both real-time and cumulative measurements alongside avatar skeleton figures. Measurements include head-height offset, proximity, number and position of touches, voice recognition, real-time static pose recognition, as well as a rudimentary overall 'Q' factor for the session. A second Kinect sensor allows continuous measurements if the first Kinect's view is blocked. This also opens up the possibility for a 'second opinion': the analysis from the two sensors can be compared and the measurement with the highest confidence selected. Currently, the software modules are being rewritten to encompass the recently released second generation Kinect sensor, which has higher resolution and better feature recognition.

It is envisaged that as the speech-pathology intervention progresses the automatically generated quality factors from the toolbox will show a marked improvement. This would also independently validate the intervention process. The toolbox could be used in future iterations as an 'expert system' that would provide speech-pathology support in areas that are typically underserved.

The project also includes the testing of technologies proposed for remote delivery of the Hanen More Than Words parent education program. For example, various videoconferencing tools on PC's, laptops and iPads have been trialled along with specialised video recording software that captures interactions for subsequent review in the education sessions. Trials of the selected technology have started with a small number of volunteer families, and their feedback will form part of the project outcomes.


  • Patricia Eadie - Audiology and Speech Pathology
  • Bronwyn Davidson - Audiology and Speech Pathology
  • Robyn Garnett - Audiology and Speech Pathology
  • Ken Clarke - Melbourne Networked Society Institute
  • Zaher Joukhadar - Melbourne Networked Society Institute


IBES Seed Funding 2013

Expected Completition

November 2015