Collaborative Research: SCH: Multimodal Algorithms for Motor Imitation Assessment in Children with Autism

About This Grant

Approximately 1 in 54 children in the US is diagnosed with autism spectrum disorder (ASD). Given its high prevalence, there is a need for an automatic and scalable method to inform diagnosis and behavioral therapies. While prior work on finding early-emerging and reliable quantitative biomarkers of ASD has focused on non-motor features, abundant research evidence has revealed patterns of impaired motor imitation in a wide range of children with ASD, making motor imitation deficits a promising avenue to find a phenotypic biomarker. However, traditional imitation assessment methods often rely on expert-based observation, which is costly, time-consuming and error-prone, and lacks objectivity and scalability. Recent advances in computer vision and machine learning make artificial intelligence a promising technology to design an objective, reproducible and highly-scalable multimodal system functioning not only in well-equipped clinical setups but also at home for assessing imitation performance in children with ASD. However, critical challenges such as the design of specific imitation tasks for ASD assessment, the collection and labeling of multimodal data for training machine learning algorithms, and the development of novel fine-grained representations human movements and metrics for comparing such movements need to be addressed to test the validity, scalability and reproducibility of automatic motor imitation assessment algorithms to inform ASD diagnosis. The overall goal of this project is to design, develop and test an objective, reproducible and highly-scalable multimodal system to observe children performing a brief video game-like motor imitation task, quantitatively assess their motor imitation performance, and investigate its validity as a phenotypic biomarker for autism. Accomplishing this goal will require an interdisciplinary approach which combines expertise in autism, child development, computer vision and machine learning. Specifically, this project will: (1) design motor imitation tasks that are relevant for ASD assessment, (2) design, test and validate a scalable and flexible system to collect and label multimodal data of children imitating a sequence of movements; (3) design a novel fine-grained representation of human movements that can be learned efficiently and is suitable for comparing the children's movements to the movements they need to imitate; (4) develop novel computer vision and metric learning algorithms for learning and comparing multimodal representations of human movements, and (5) use such metrics to generate candidate imitations scores that can be used as potential quantitative biomarkers for ASD. The motor imitation assessment methods to be developed in this project could be used in a wide variety of applications beyond assessing children with ASD, such as providing imitation performance scores for video-based rehabilitation therapy, surgical skill assessment, athletic activities and other movement-based instructional activities. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.