One of the most prevalent illnesses in children is autism spectrum disorder (ASD) (1 in 44). According to some estimates, 53% of kids with ASD engage in one or more challenging behaviors (CB; aggression, self-injury, property destruction, elopement, etc.), which is significantly higher than the prevalence among their peers who are typically developing or who have other developmental disorders. Numerous, significant negative effects of CB on the person exist, and they are linked to a worse long-term outlook. For caregivers of children with ASD, the presence of CB is a better indicator of stress than the severity of the child's core ASD symptoms. The validity of fixed features extracted from autistic children’s face photographs as a biomarker to demarcate them from healthy children is investigated in this study paper. The proposed paper aims to use deep learning models (CNN) to classify autism spectrum disorders based on facial expression images. By leveraging the power of deep convolutional neural networks, based on the Kaggle dataset. We used and prepared data input to CNN models where the split image in two parts horizontally and vertically as feature extractor’s model as a binary classifier to identify autism in children accurately. Our results reveal that the proposed model achieved an accuracy of 94%, Sensitivity of 93% and Specificity of 95% this indicator is considered important and can be built or relied on.

 Keywords: computer vision, machine learning, CNN, autism spectrum disorder, behavioral science

Autism spectrum disorder (ASD) is a set of neurodevelopmental disorders characterized by a lack of verbal and nonverbal communication in the first three years of life. Eye contact avoidance, emotional management or empathy disorders, and a conspicuously limited variety of interests and activities are just a few of the atypical social behaviors.¹ ASD is now present in between 1% and 2% of the population, according to recent large-scale surveys.^2,3 ASD has become more frequent in the last two decades.⁴

Although the average age of diagnosis has dropped and the DSM diagnostic criteria have altered, an increase in risk factors cannot be fully ruled out.^5,6 Men experience ASD two to three times more commonly than women do, according to research.^2,3,7 The under recognition of female ASDs may be the cause of this diagnostic bias in favor of males.⁸ Additionally, some studies have hypothesized that protective benefits against ASD that are particular to women may occur.⁹ Figure 1 shows the main gene and another effective factors for the ASD.

Figure 1 Schematic diagram for the main causes of ASD, the information of all class is taken from.¹⁰

Autism was previously classified into four categories by the Diagnostic and Statistical Manual of Mental Disorders-4 (DSM-4): Asperger Syndrome, Autistic Disorder, Pervasive Developmental Disorder Not Otherwise Specified (PDD-NOS), and Childhood Disintegrative Disorder (CDD).The Diagnostic and Statistical Manual of Mental Disorders-5 (DSM-5) has combined these four classifications into the diagnosis of autism spectrum disorder (ASD). ASD was classified by severity levels in the DSM-5 based on limitations in social communication and repetitive and limited conduct. The severity levels are: Level 1 - Support Required, Level 2 - Substantial Support Required, and Level 3 - Very Substantial Support Required.

Despite the fact that doctors employ standardized diagnostic instruments for ASD diagnosis, one significant limitation of the approach is that applying diagnostic tools necessitates a significant amount of time “to complete the evaluation and interpret the results.¹¹ This issue has a clever machine learning approach suggested as a solution. The main goal of machine learning research for ASD detection is to increase diagnostic accuracy while reducing diagnostic time. Reduced diagnosis time allows for rapid intervention for ASD patients. By reducing the dimensionality of the relevant input information, the machine learning technique also aims to identify the top ASD traits.

The study suggests that the random SVM cluster method has potential for assisting in the auxiliary diagnosis of ASD, based on resting-state functional magnetic resonance imaging (fMRI) data.¹² The researcher using brain imaging data from the Autism Brain Imaging Data Exchange (ABIDE) dataset to identify ASD. Here used Multilayer perceptron with backpropagations algorithm.¹³ The paper discusses mobile autism risk assessment tools. Designed for mobile devices, the program identifies kids at risk of autism spectrum condition early. They use binary firefly algorithm, they got 91-92% accuracy.¹⁴ The researchers used crowdsourcing to get information. They gathered a lot of clinical tests and behavioral observations from people with autism and ADHD, as well as from people who were growing normally.  They use SVM algorithm with 60 to 90% accuracy.¹⁵ These studies used SVM random algorithm with an accuracy of 89%.¹⁶⁻¹⁸ The research provides a machine-learning approach to predict autism symptoms in any age. The research presents a more efficient and accurate machine learning approach by mixing Random Forest-CART and Random Forest-ID3 algorithms to predict autism. A smartphone app based on this model was created. The suggested model was tested using the AQ-10 dataset and 250 real datasets from autistic and non-autistic people.¹⁹ The paper investigates graph representation learning methods and deep neural networks to detect ASD patients using brain functional connectivity characteristics. AWE, Node2vec, Struct2vec, multi-node2vec, and Graph2Img represent the ABIDE I and II databases, with Graph2Img being the best. The study achieves 80% accuracy for leave-one-site-out cross-validation using PCA for feature vector extraction and latent deep neural network for classification.²⁰ This study presents a practical method for screening Autism Spectrum Disorder (ASD) using face images. We utilize VGG16 transfer learning-based deep learning techniques on a distinct dataset of clinically diagnosed children with ASD that we have gathered.²¹ In this work introduced a deep learning model, Mobile Net, which uses deep learning to classify children into healthy children or potential autistic children according to their facial images. The dataset used in this research is from Kaggle.²² Using a Deep Convolutional Neural Network (CNN) to classify facial emotions in people with and without Autism Spectrum Disorder (ASD) using electroencephalography (EEG) signals. The study discovered that CNN successfully identified facial emotions for both ASD and non-ASD groups, implying that facial emotion information is present in the brain signal of people with ASD.²³ The paper talks about how machine learning techniques could be used to find autism spectrum disorder (ASD) automatically using brain imaging data. It talks about how machine learning methods can be used to find ASD early on and the problems when combining machine learning with brain imaging data from different places.²⁴ This work employed transfer learning with MobileNetV2 and MobileNetV3-Large models trained on ImageNet to recognize and identify facial expressions in autistic children.²⁵ The researchers created an autism classifier using deep learning techniques. They trained a computer vision model using video data from a mobile game application.²⁶ The paper discusses the advancement of artificial intelligence (AI) models in diagnosing autism spectrum disorder (ASD) through analyzing facial expressions and clues. The paper investigates integrating convolutional neural network (CNN) models with XGBoost and RF algorithms in hybrid systems.²⁷ The paper discusses creating a real-time system that can identify emotions in autistic children by utilizing deep learning and Internet of Things (IoT) technologies. The system's objective is to identify emotions in autistic children and to aid in the management of pain or rage. The document also emphasizes the significance of assistive technology in enhancing the quality of life for those with autism. The text underscores the importance of choosing suitable assistive technology that aligns with individual requirements and features. In addition, the document discusses using facial expression recognition for the early detection of autism.²⁸ The suggested paper plans to use deep learning models (CNN) to classify images of facial expressions for people with autism spectrum disorders and Non-ASD. We used and prepared data to be fed into CNN models. The image was split horizontally and vertically to use as a feature extractor and a binary classifier to correctly find children with autism.  In this paper, we present a convolutional neural network classifier for the classification of ASD and non-ASD. The rest of the paper is organized as: section 3 presents the proposed system modeling and the main database, section 4 presents experiment, section 5, 6 is the discussion and conclusion.

Proposed method              

CNNs have achieved remarkable success in various computer vision tasks, surpassing traditional methods in many cases. Their ability to automatically learn hierarchical features from raw input data makes them highly effective at extracting meaningful information from images and videos. A convolutional neural network is a deep learning algorithm that processes and analyzes visual data, such as images or videos. It is widely used in computer vision tasks, including image classification, object detection, and image segmentation. They consist of multiple layers, including convolutional, pooling, and fully connected layers. The convolutional layers extract features from input images by applying convolutional filters.

These filters detect patterns and edges in the images, gradually learning more complex features as the network goes deeper. Pooling layers are used to down-sample the feature maps obtained from the convolutional layers. They reduce the spatial dimensions of the feature maps, helping to extract the most essential features and providing some degree of translation invariance. Finally, fully connected layers are used for classification or regression tasks.

They take the high-level features learned from the previous layers and map them to the desired output classes or values. CNNs have achieved remarkable success in various computer vision tasks, surpassing traditional methods in many cases. Their ability to automatically learn hierarchical features from raw input data makes them highly effective at extracting meaningful information from images and videos. Figure 2 displays Flowchart of the proposed system.

Figure 2 Flowchart of the proposed system.

Dataset

The dataset used in the training and testing of this model was captured from the Kaggle website, which contains 2,936 face images. This is the only dataset available to the public. In this work, we chose of 1468 images for each label (autistic and non-autistic). The dataset consists of facial images of autistic and non-autistic children.

The images include boys and girls, and the number of images of non-autistic children is equal to that of autistic children. The sizes of these images in the dataset are different, so they must be converted to a unified specification when training a model. In this research, the intermediate domain of four-phase deep learning is facial expression recognition of ordinary people, and the size of the image used in this task is 256 × 256. It was divided into two parts (autistic and non-autistic) in training and test sets. The training set had 80% of the images in the whole dataset. The test and validation sets contain 0.1 for each one, respectively. The images were cropped and cleaned up during the data preparation process. The normalizing method was applied to the scale before the deep learning model could be trained on the collected data.

Facial recognition using Convolutional Neural Networks (CNNs) has seen significant research, focusing on various aspects such as architectural design and loss function design. Here, we used a six-layer CNN architecture with six convolutional layers and one fully connected layer. In their method, a CNN architecture for end-to-end feature learning and classification follows this preprocessing step, as shown in Figure 3.

Figure 3 A framework of the proposed classification system.

In this part, we proposed a method for the classifiers that used the original image size with a partitioned image size of half vertically and horizontally as input to CNN.

First case (Scenario)

Initially, we put the original images in two folders (half for autistic and another half for non-autistic) as classification labels. In this approach, we used two folders to apply two parallel convolutional neural networks to classify and predict autistic and non-autistic, as shown in Figure 4.

Figure 4 A framework of the first case.

Second case (Scenario)

In this case, we took the images. We divided them horizontally into two parts, where there are two folders, one containing the top part of the image and the other containing the bottom part image, as in the previous first case, as shown in Figure 5.

Figure 5 A framework of the second case.

Third case (Scenario) 

According to the third case, the same second case was used here, but the images were used vertically rather than horizontally. In this case, the image is cut into two portions. Figure 6 displays the convolutional neural network.

Figure 6 A framework of the third case.

Sensitivity, specificity, accuracy, recall, and F1-score were among the evaluation measures used to assess how well the supplied models identified ASDs. The following are the equations of these parameters:

$Accuracy= \frac{TP+TN}{TP+FP+FN+TN}$   (1)

$Sensitivity= \frac{TP}{TP+FN}$   (2)

$Precision= \frac{TP}{TP+FP}$   (3)

$Specificity= \frac{TN}{FP+TN}$   (4)

$F1-score =2\times \frac{precision\times Sensitivity}{precision+Sensitivity}$   (5)

Data augmentation is a more powerful support technique used in machine learning and computer vision, involves application that used to classification and detection ASD. The process entails implementing diverse modifications to the initial dataset to generate supplementary augmented samples. Augmentation enhances the variety of the data, hence mitigating overfitting and enhancing the model's capacity to generalize. Facial feature analysis for individuals with autism benefits greatly from augmentation, as it enables the model to learn from a wider range of photos, resulting in improved performance and more reliable outcomes. The dataset comprised 2936 face feature photos obtained from individuals with autism spectrum disorder (ASD) and persons without ASD. There were 1468 images in each class, one for ASD and one for Non-ASD.

The MATLAB application was utilized to assist in the training of the learning models. A small selection of the data visualization and analysis tools that were used to assess the models' efficacy. We compared the performance of three case models trained with different optimizers using a batch size of 90 and a learning rate of 0.0005 over 70 epochs using a standard set of hyperparameters with the following values. There were 2936 photos in all in the dataset. As shown in Table 1, we gave an image a value of 0 for kids in the non-ASD group and a value of 1 for kids with an ASD diagnosis when we were creating the data frame. It is noted that the accuracy of the first case model was 90%, the second case model was 94%, and the third case model was 84%.

The confusion matrix in Figure 7 displays the true-negative and false-positive rates, as well as the valid-positive and false-negative indications for the first, second, and third cases. Based on the evaluation measures, the second case model demonstrated commendable accuracy outcomes.

Figure 7 Confusion matrix of the third case.

The three scenarios of loss train and validate are shown graphically in Figure 8, with each iteration of 10 epochs (X-axis) and the model's percentage loss expressed in % (Y-axis). We looked at the validation system's performance to get a precise picture of how well the training system worked. A pause in the optimization process resulted in a notable improvement in precision, which was finally increased to 70 epochs.

Figure 8 Performance of the third case (Loss during train and validate).

Face expressions or emotions could help identify ASD. Working with kids requires a clear and unambiguous ASD diagnosis to address issues. Because ASD patients have complex attentional behaviors, making these tools is difficult. Autism Spectrum Disorder (ASD) sufferers benefit most from early detection and treatment. ASD diagnosis in children has traditionally required hospitalization, which is expensive and time-consuming. This study presents an impartial, practical, and effective way for diagnosing Autism Spectrum Disorder (ASD) in children by evaluating their facial expressions and emotions.

We bridge the gap between autism categorization and facial analysis, making automated autism categorization cheaper and faster. Using diverse imaging situations, our deep convolutional neural network model achieves both goals.

The model was trained and verified using 2940 photos, 80% for training and 20% for validation and testing. Both autistic and non-autistic children provided data. Our study found that a divided image may be enough to diagnose autism in children. This diagnostic approach may work for different diseases. Your findings reveal that the proposed model achieved an accuracy of 94%. The results of the model compared to different existing systems are presented in Table 2.²⁹⁻³¹

This study looked into the utility of facial features as a biomarker for accurately distinguishing autistic children from non-ASD children. We used a publicly available dataset that included face images from both children (ASD and non-ASD). Three models of CNN-based binary ASD classifier models that use three different cases, were built and tested to evaluate the scores of each model. We discovered that used the horizontally half divided face model outperformed the other models and achieved accuracy 94%, a Specificity of 95 %, and an F1-Score of 94%. The results indicate that specific features of autism spectrum disorder (ASD) may be effectively extracted from still images of a child's face, enabling a rapid and precise method for screening ASD.

None.

Authors declare that there is no conflict of interest.

1. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Arlington VA, 3rd edn. American Psychiatric Association; 1980. 
2. Mattila ML, Kielinen M, Linna SL, et al. Autism spectrum disorders according to DSM-IV-TR and comparison with DSM-5 draft criteria: an epidemiological study. J Am Acad Child Adolesc Psychiatry. 2011;50(6):583–592.
3. Kim YS, Leventhal BL, Koh YJ, et al. Prevalence of autism spectrum disorders in a total population sample. Am J Psychiatry. 2011;168(9):904–912.
4. Fisch GS. Nosology and epidemiology in autism: classification counts. Am J Med Genet C Semin Med Genet. 2012;160(2):91–103.
5. Elsabbagh M, Divan G, Koh YJ, et al. Global prevalence of autism and other pervasive developmental disorders. Autism Res. 2012;5(3):160–179.
6.  Fombonne E. Incidence and prevalence of pervasive developmental disorders. In: Hollander E, Kolevzon A, Coyle JT, editors. Textbook of autism spectrum disorders. Washington, DC: American Psychiatric Publishing; 2012. 117–136 p.
7. Saemundsen E, Ludvigsson P, Rafnsson V. Autism spectrum disorders in children with a history of infantile spasms: a population-based study. J Child Neurol. 2007;22(9):1102–1107.
8. Baron-Cohen S, Lombardo MV, Auyeung B, et al. Why are autism spectrum conditions more prevalent in males? PLoS Biol. 2011;9(6):e1001081.
9. Robinson EB, Lichtenstein P, Anckarsäter H, et al. Examining and interpreting the female protective effect against autistic behavior. Proc Natl Acad Sci U S A. 2013;110(13):5258–5262. 
10. Hye Ran P, Jae Meen L, Hyo Eun M, et al. A short review on the current understanding of autism spectrum disorders. Exp Neurobiol. 2016;25(1):1–13.
11.  Lai MC, Lombardo MV, Baron-Cohen S. “Autism”. Lancet. 2014;383(9920):896–910.
12.  Bi XA, Wang Y, Shu Q, et al. Classification of autism spectrum disorder using random support vector machine cluster. Front Genet. 2018;9:18.
13. Heinsfeld AS, Franco AR, Craddock RC, et al. Identification of autism spectrum disorder using deep learning and the ABIDE dataset. Neuroimage Clin. 2018;17:16–23.
14. Duda M, Daniels J, Wall DP. Clinical evaluation of a novel and mobile autism risk assessment. J Autism Dev Disord. 2018;46(6):1953–1961.
15.  Duda M, Haber N, Daniels J, et al. Crowdsourced validation of a machine-learning classification system for autism and ADHD. Transl. Psychiatry. 2017;7(5):e1133.
16. Achenie LE, Scarpa A, Factor RS, et al. A machine learning strategy for autism screening in toddlers. J Dev Behav Pediatr. 2019;40(5):369–376.
17. Levy S, Duda M, Haber N, et al. Sparsifying machine learning models identify stable subsets of predictive features for behavioral detection of autism. Mol Autism. 2017;8:65.
18. Wang C, Xiao Z, Wu J. Functional connectivity-based classification of autism and control using SVM-RFECV on rs-fMRI data. Phys Med. 2019;65:99–105.
19. Omar KS, Mondal P, Khan NS, et al. A lachine learning approach to predict autism spectrum disorder. 2019 International Conference on Electrical, Computer and Communication Engineering (ECCE). 2019:1–6 p.
20. Sharma M. Improved autistic spectrum disorder estimation using Cfs Subset with greedy stepwise feature selection technique. Int J Inf Technol. 2019;14:1251–1261.
21. Lu A, Perkowski M. Deep learning approach for screening autism Spectrum disorder in children with facial images and analysis of Ethnoracial factors in model development and application. Brain Sci. 2021;11(11):1446.
22. Hosseini MP, Beary M, Hadsell A, et al. Deep learning for autism diagnosis and facial analysis in children. Front Comput Neurosci. 2022:789998.
23. Juan Manuel Mayor T, Tessa C, Kathryn M. et al. Facial emotions are accurately encoded in the neural signal of those with autism spectrum disorder: A deep learning approach. Biol Psychiatry: Cogn Neurosci Neuroimaging. 2022;7(7):688−695.
24. Hamza S, Rizwan Ahmed K. A novel machine learning based framework for detection of autism spectrum disorder (ASD). Applied Artificial Intelligence. 2022;36:1.
25. Li Y, Huang WC, Song PH. A face image classification method of autistic children based on the two-phase transfer learning. Front Psychol. 2023;14:1226470.
26. Sleiman E, Mutlu Onur C, Surabhi S, et al. Deep learning-based autism spectrum disorder detection using emotion features from video rcordings. JMIR Biomed Eng. 2022;7(2):e39982)
27. Awaji B, Senan EM, Olayah F, et al. Hybrid techniques of facial feature image analysis for early detection of autism spectrum disorder based on combined CNN features. Diagnostics (Basel). 2023;13(18):2948.
28. Talaat FM. Real-time facial emotion recognition system among children with autism based on deep learning and IoT. Neural Comput Applic. 2023;35:12717–12728.
29. Mujeeb Rahman KK, Subashini MM. Identification of autism in children using static facial features and deep neural networks. Brain Sci. 2022;12(1):94.
30. Fadi T, Firuz K, Khairan R. A new computational intelligence approach to detect autistic features for autism screening. International Journal of Medical Informatics. 2018;117:112−124.
31. Alkahtani H, Aldhyani THH, Alzahrani MY. Deep learning algorithms to identify autism spectrum disorder in children-based facial landmarks. Appl Sci. 2023;13(8):4855.