AutoPlay Smart Toys Kit for Autism Spectrum Disorder (ASD)

Early interactions of children with autism spectrum disorder (ASD) exhibit distinctive motor and object-manipulation features that may emerge within the first three years of life. This Master’s thesis presents a proof of concept for using instrumented toys and machine-learning algorithms for digital behavioral phenotyping and early screening of ASD. The project was conducted within the AutoPlay initiative, with the aim of providing clinicians with an objective decision-support system that integrates traditional assessments (e.g., ADOS-2) with quantitative, unobtrusive data. Unlike approaches based on wearable sensors or tablet interactions, the system employs a kit of commonly used toys equipped with inertial measurement units (IMUs) to record acceleration, angular velocity, and spatial orientation during free play in a naturalistic setting. The work comprises a comprehensive review of the clinical context of ASD and a detailed description of the data-collection, preprocessing, and analysis methodology. The analyzed dataset consists of 20 recordings of children younger than 30 months (12 male, 8 female) from diverse nationalities backgrounds. Of these, 14 were classified as high risk (ADOS > 14) and 6 as medium–low risk (ADOS ≤ 14), following opportunistic recruitment. Play sessions were video-recorded at EOC Hospital, Bellinzona (Switzerland) and manually annotated, enabling extraction and classification of children’s actions into several categories, including those based on play type (Exploratory, Functional, Symbolic) and kinematics (Linear, Rotational, Forceful, Handling). Signals are segmented into overlapping time windows and described using a set of statistical and dynamic features; four supervised classifiers (Decision Tree, k-Nearest Neighbors, Random Forest, Support Vector Machine) were evaluated with and without z-score normalization. All analyses were carried out in MATLAB. Results for the action-recognition model show modest accuracy (approximately 30–35%), with a strong tendency to confuse actions with a broad dominant class (“Holds in hand”). Similarly, binary classification of ASD risk (high vs. medium–low) attains a maximum accuracy of 60% but insufficient sensitivity for the lower-risk class. These limitations possibly depend to intrinsic dataset limitations, including small sample size, class imbalance, inconsistencies in data collection, and ambiguities in manual labeling. In conclusion, although the current results are insufficient for direct clinical application, this thesis validates the methodological approach and delineates a clear roadmap for future work. Priorities include improving the data-collection protocol, expanding and balancing the sample, implementing advanced machine-learning techniques to address class imbalance, and exploring deep-learning models capable of capturing temporal dependencies in behavioral signals. While not a definitive diagnostic solution, the study provides a critical analysis that demonstrates the potential of biomedical engineering for quantitative observation of ASD symptoms in ecologically valid settings and establishes a robust foundation for a long-term research program.