Deep learning algorithms for autism spectrum disorder detection using eye-tracking patterns
DOI:
https://doi.org/10.21107/kursor.v13i3.468Keywords:
Autism, computer vision, deep learning model, vision transformerAbstract
Early diagnosis of autism spectrum disorder (ASD) plays a crucial role in facilitating prompt interventions and assessing post-therapy progress, both of which can greatly improve developmental trajectories. The emergence of artificial intelligence—particularly deep learning—has opened new possibilities for clinicians to detect ASD with improved precision and speed. This study investigates the utilization of deep learning models to recognize ASD among 13-year-old children based on eye movement data collected as participants observed static images and short video sequences. The dataset included visual and numerical variables, such as gaze position and pupil diameter, allowing for a multimodal analytical approach. For the numerical dataset, a multilayer perceptron (MLP) neural network produced the best outcomes, yielding an accuracy of 91.7% and a recall rate of 83.3% in ASD classification. Meanwhile, the Vision Transformer (ViT) model performed best for image-based analysis, reaching an accuracy of 78.2% and a recall rate of 88.6%. Overall, the findings highlight the potential of deep learning frameworks as objective, data-driven tools for ASD detection in both clinical and research contexts.
Key words: Autism, Computer Vision, Deep Learning Model, Vision Transformer.
Downloads
References
[1] American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders: DSM-5. Arlington, VA, USA: American Psychiatric Publishing, 2013. https://doi.org/10.1176/appi.books.9780890425596.
[2] M. S. Aljehany and K. D. Bennett, “A comparison of video prompting to least-to-most prompting among children with autism and intellectual disability,” J. Autism Dev. Disord., vol. 50, no. 5, pp. 1714–1724, 2020, doi: 10.1007/s10803-019-03929-x.
[3] A. Farrell, “A spectrum of inclusion: Strategies for students with ASD,” Agora, vol. 54, no. 2, pp. 40–43, 2019. [Online]. Available: https://www.proquest.com/scholarly-journals/spectrum-inclusion-strategies-students-with-asd/docview/2599639618/se-2
[4] V. Hees, T. Moyson, and H. Roeyers, “Higher education experiences of students with autism spectrum disorder: Challenges, benefits and support needs,” J. Autism Dev. Disord., vol. 45, pp. 1673–1688, 2015, doi: 10.1007/s10803-014-2324-2.
[5] Y.-W. Nah and J. W.-L. Tan, “Brief report: The effect of gender on teachers’ perceptions of behaviours in students with autism spectrum disorder (ASD),” Educ. Dev. Psychol., vol. 38, no. 2, pp. 249–254, 2021, doi: 10.1080/20590776.2021.1985378.
[6] M. Dean, C. Kasari, W. Shih, F. Frankel, R. Whitney, R. Landa, et al., “The peer relationships of girls with ASD at school: Comparison to boys and girls with and without ASD,” J. Child Psychol. Psychiatry, vol. 55, no. 11, pp. 1218–1225, 2014, doi: 10.1111/jcpp.12242.
[7] T. Ma, H. Wu, and Y. Wang, “Application of different treatment methods for autism,” in Proc. 3rd Int. Conf. Biol. Eng. Med. Sci., 2023, doi: 10.54254/2753-8818/20/20230705.
[8] R. Lordan, C. Storni, and C. A. De Benedictis, Autism Spectrum Disorders: Diagnosis and Treatment. Brisbane, Australia: Exon Publications, 2021. https://doi.org/10.36255/exonpublications.autismspectrumdisorders.2021.diagnosis
[9] S. Wyver, “Book review: Autism spectrum disorders: Interventions and treatments for children and youth,” Autism, vol. 10, pp. 118–119, 2006. https://doi.org/10.1177/136236130601000110
[10] M. De, R. Chakraborty, M. Bhattacharya, and I. Basu, “Challenges for teachers in inclusive education vis-à-vis the barriers faced by students with autism spectrum disorder (ASD),” Harvest (Online), vol. 3, no. 1, 2018. [Online]. Available: https://www.harvestjournal.net/wp-content/uploads/2018/07/Mitu-De.pdf
[11] J. M. Gillis and R. Beights, “New and familiar roles for clinical psychologists in the effective treatment for children with an autism spectrum disorder,” Cogn. Behav. Pract., vol. 19, no. 3, pp. 392–400, 2012, doi: 10.1016/j.cbpra.2011.02.007.
[12] R. Simeoli, A. Rega, M. Cerasuolo, R. Nappo, and D. Marocco, “Using machine learning for motion analysis to early detect autism spectrum disorder: A systematic review,” Rev. J. Autism Dev. Disord., 2024, doi: 10.1007/s40489-024-00435-4.
[13] M. Jiang, S. M. Francis, D. Srishyla, C. Conelea, Q. Zhao, and S. Jacob, “Classifying individuals with ASD through facial emotion recognition and eye-tracking,” in Proc. 2019 41st Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBC), 2019, pp. 6063–6068, doi: 10.1109/EMBC.2019.8857005.
[14] G. Wan, X. Kong, B. Sun, S. Yu, Y. Tu, J. Park, C. Lang, M. Koh, Z. Wei, Z. Feng, Y. Lin, and J. Kong, “Applying eye tracking to identify autism spectrum disorder in children,” J. Autism Dev. Disord., vol. 49, no. 1, pp. 209–215, 2019, doi: 10.1007/s10803-018-3690-y.
[15] F. Cilia, R. Carette, M. Elbattah, G. Dequen, J.-L. Guérin, J. Bosche, L. Vandromme, and B. Le Driant, “Computer-aided screening of autism spectrum disorder: Eye-tracking study using data visualization and deep learning,” JMIR Hum. Factors, vol. 8, no.4, p. e27706, 2021, doi: 10.2196/27706.
[16] K. J. Tsuchiya, S. Hakoshima, T. Hara, M. Ninomiya, M. Saito, T. Fujioka, H. Kosaka, Y. Hirano, M. Matsuo, M. Kikuchi, Y. Maegaki, T. Harada, T. Nishimura, and T. Katayama, “Diagnosing autism spectrum disorder without expertise: A pilot study of 5-to 17-year-old individuals using Gazefinder,” Front. Neurol., vol. 11, p. 603085, 2021, doi: 10.3389/fneur.2020.603085.
[17] M. R. Kanhirakadavath and M. S. M. Chandran, “Investigation of eye-tracking scan path as a biomarker for autism screening using machine learning algorithms,” Diagnostics, vol. 12, no. 2, p. 518, 2022, doi: 10.3390/diagnostics12020518.
[18] W. Liu, M. Li, and L. Yi, “Identifying children with autism spectrum disorder based on their face processing abnormality: A machine learning framework,” Autism Res., vol. 9, no. 8, pp. 888–898, 2016, doi: 10.1002/aur.1615.
[19] M. Elbattah, R. Carette, G. Dequen, J.-L. Guérin, and F. Cilia, “Learning clusters in autism spectrum disorder: Image-based clustering of eye-tracking scanpaths with deep autoencoder,” in Proc. 2019 41st Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBC), 2019, pp. 1417–1420, doi: 10.1109/EMBC.2019.8856904.
[20] Z. Zhao, H. Tang, X. Zhang, X. Qu, X. Hu, and J. Lu, “Classification of children with autism and typical development using eye-tracking data from face-to-face conversations: Machine learning model development and performance evaluation,” J. Med. Internet Res., vol. 23, no. 8, p. e29328, 2021, doi: 10.2196/29328.
[21] S. Liaqat, C. Wu, P. R. Duggirala, S. S. Cheung, C.-N. Chuah, S. Ozonoff, and G. Young, “Predicting ASD diagnosis in children with synthetic and image-based eye gaze data,” Signal Process. Image Commun., vol. 94, p. 116198, 2021, doi:10.1016/j.image.2021.116198.
[22] M. Alcañiz, I. A. Chicchi-Giglioli, L. A. Carrasco-Ribelles, J. Marín-Morales, M. E. Minissi, G. Teruel-García, M. Sirera, and L. Abad, “Eye gaze as a biomarker in the recognition of autism spectrum disorder using virtual reality and machine learning: A proof of concept for diagnosis,” Autism Res., vol. 15, no. 1, pp. 131–145, 2022, doi: 10.1002/aur.2636.
[23] Y. Lin, Y. Gu, Y. Xu, S. Hou, R. Ding, and S. Ni, “Autistic spectrum traits detection and early screening: A machine learning-based eye movement study,” J. Child Adolesc. Psychiatr. Nurs., vol. 35, no. 1, pp. 83–92, 2022, doi: 10.1111/jcap.12346.
[24] C. M. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Clarendon Press, 1995. https://doi.org/10.1093/oso/9780198538493.001.0001
[25] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, 1998, doi: 10.1109/5.726791.
[26] Swastika, W., Prilianti, K. R., Irawan, P. L. T., and Setiawan, H., “Comparative analysis of random forest and deep learning approaches for automated acute lymphoblastic leukemia detection using morphological and textural features,” Jurnal Ilmiah Kursor, vol. 13, no. 1, Jul. 2025. doi: 10.21107/kursor.v13i1.427.
[27] Putri, F. N. R., Isnanto, R. R., & Sugiharto, A., “Skin Rash Classification System Using Modified DenseNet201 through Random Search for Hyperparameter Tuning,” Jurnal Ilmiah Kursor, vol. 12, no. 4, Dec. 2024. doi: 10.21107/kursor.v12i4.418.
[28] Badri, F., Alawiy, M. T., & Yuniarno, E. M., “Deep Learning Architecture Based on Convolutional Neural Network (CNN) in Image Classification,” Jurnal Ilmiah Kursor, vol. 12, no. 2, pp. 83–92, Dec. 2023, doi: 10.21107/kursor.v12i2.349.
[29] Sabila, S. S. and Tjahyaningtyas, H. P. A., “The Multiple Brain Tumor with Modified DenseNet121 Architecture Using Brain MRI Images,” Jurnal Ilmiah Kursor, vol. 12, no. 3, 2024, doi: 10.21107/kursor.v12i3.379.
[30] Hasan Basri, I. Syarif, S. Sukaridhoto, and M. F. Falah, “Intelligent System for Automatic Classification of Fruit Defect Using Faster Region-Based Convolutional Neural Network (Faster R-CNN),” Jurnal Ilmiah Kursor, vol. 10, no. 1, Jul. 2019, doi: 10.28961/kursor.v10i1.187.
[31] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. 2016 IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2016, pp. 770–778, doi: 10.1109/CVPR.2016.90.
[32] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proc. 2017 IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2017, pp. 2261–2269, doi: 10.1109/CVPR.2017.243.
[33] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in I. Guyon, U. Von Luxburg,
S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., Adv. Neural Inf. Process. Syst., vol. 30. Red Hook, NY, USA: Curran Associates, Inc., 2017.
[34] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020. [Online]. Available: https://arxiv.org/abs/2010.11929 .
[35] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin Transformer: Hierarchical vision transformer using shifted windows,” arXiv preprint arXiv:2103.14030, 2021. [Online]. Available: https://arxiv.org/abs/2103.14030 .
[36] F. Cilia, R. Carette, M. Elbattah, J.-L. Guérin, and G. Dequen, “Eye-tracking dataset to support the research on autism spectrum disorder,” Figshare, 2022, doi: 10.6084/m9.figshare.20113592.v1.
[37] E. Schopler, R. J. Reichler, and B. R. Renner, The Childhood Autism Rating Scale (CARS) Manual. Los Angeles, CA, USA: Western Psychological Services, 1988.
[38] R. Carette, M. Elbattah, G. Dequen, J.-L. Guérin, and F. Cilia, “Visualization of eye-tracking patterns in autism spectrum disorder: Method and dataset,” in Proc. 2018 13th Int. Conf. Digit. Inf. Manage. (ICDIM), 2018, pp. 248–253, doi: 10.1109/ICDIM.2018.8846967.
[39] S. S. Dcouto and J. Pradeepkandhasamy, “Multimodal deep learning in early autism detection—Recent advances and challenges,” Eng. Proc., vol. 59, no. 1, p. 205, 2023, doi: 10.3390/engproc2023059205.
[40] N. Neethu, K. R. Remya, and B. M. Varghese, “Early detection of autism spectrum disorder via deep-learning application of fMRI and machine learning for ASD children identifications,” 2024, pp. 1–15.
Downloads
Published
Issue
Section
Citation Check
License
Copyright (c) 2026 Muhamad Syukron, Rangga Alif Faresta
This work is licensed under a Creative Commons Attribution 4.0 International License.
+62-858-5000-1918
