A Novel Dataset for Video-Based Autism Classification Leveraging Extra-Stimulatory Behavior

Read original: arXiv:2409.04598 - Published 9/10/2024 by Manuel Serna-Aguilera, Xuan Bac Nguyen, Han-Seok Seo, Khoa Luu
Total Score

0

A Novel Dataset for Video-Based Autism Classification Leveraging Extra-Stimulatory Behavior

Sign in to get full access

or

If you already have an account, we'll log you in

Overview

  • A new dataset for video-based autism classification that leverages extra-stimulatory behavior
  • Aims to improve autism detection by analyzing non-verbal cues and behaviors beyond just social interaction
  • Proposes a novel framework for classifying autism from video data

Plain English Explanation

The paper discusses a new dataset for detecting autism spectrum disorder (ASD) using video data. Traditionally, autism diagnosis has relied heavily on evaluating social interactions and communication. However, this paper argues that analyzing other types of behavior, such as extra-stimulatory behaviors (repetitive movements, sensory-seeking actions, etc.), can provide additional insights for more accurate ASD classification.

The key idea is to capture a broader range of behaviors beyond just social cues. This expanded view of autism-related behaviors can lead to more comprehensive and reliable detection models. The proposed dataset includes video recordings of both neurotypical individuals and those with ASD, annotated with a variety of behavioral markers. By leveraging this rich data, the researchers aim to develop advanced machine learning models that can better identify the signs of autism.

Technical Explanation

The paper introduces a novel dataset for video-based autism classification that goes beyond traditional social interaction analysis. The dataset includes video recordings of both neurotypical individuals and those diagnosed with ASD, along with annotations for a range of extra-stimulatory behaviors such as repetitive movements, sensory-seeking actions, and other non-verbal cues.

The researchers argue that these additional behavioral markers, beyond just social interaction patterns, can provide a more comprehensive view of autism-related characteristics. They propose a framework for using this dataset to train advanced machine learning models for ASD classification. The key aspects of the technical approach include:

  1. Video Data Collection: The dataset was curated by recording participants in various settings, capturing a diverse range of behaviors.
  2. Behavioral Annotation: Experienced clinicians annotated the video data with a detailed taxonomy of extra-stimulatory behaviors, social interactions, and other relevant markers.
  3. Model Architecture: The researchers explore different deep learning architectures, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), to leverage the video data and behavioral annotations for ASD classification.
  4. Evaluation Metrics: The performance of the models is assessed using standard metrics like accuracy, precision, recall, and F1-score, comparing the results to existing approaches.

By incorporating this expanded view of autism-related behaviors, the proposed framework aims to enhance the accuracy and robustness of video-based ASD detection, potentially leading to earlier diagnosis and improved interventions.

Critical Analysis

The paper presents a compelling approach to improving autism detection by leveraging a novel dataset that captures a broader range of behavioral markers beyond just social interaction. This is a valuable contribution, as traditional autism diagnosis often relies heavily on evaluating social cues, which may overlook other important behavioral patterns.

However, the paper does acknowledge some limitations of the dataset, such as the relatively small sample size and the potential for bias in the video recording and annotation process. Additionally, the proposed models, while promising, may require further validation and testing to ensure their generalizability and robustness in real-world clinical settings.

It would also be beneficial to explore the potential ethical implications of using video-based autism detection, particularly regarding privacy concerns and the responsible use of such technologies.

Conclusion

This paper presents a novel dataset and framework for video-based autism classification that goes beyond traditional social interaction analysis. By incorporating a broader range of extra-stimulatory behaviors, the proposed approach aims to enhance the accuracy and robustness of ASD detection, potentially leading to earlier diagnosis and more targeted interventions.

While the research shows promise, further validation and consideration of the ethical implications are necessary to ensure the responsible development and deployment of such technologies in real-world clinical settings.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

A Novel Dataset for Video-Based Autism Classification Leveraging Extra-Stimulatory Behavior
Total Score

0

A Novel Dataset for Video-Based Autism Classification Leveraging Extra-Stimulatory Behavior

Manuel Serna-Aguilera, Xuan Bac Nguyen, Han-Seok Seo, Khoa Luu

Autism Spectrum Disorder (ASD) can affect individuals at varying degrees of intensity, from challenges in overall health, communication, and sensory processing, and this often begins at a young age. Thus, it is critical for medical professionals to be able to accurately diagnose ASD in young children, but doing so is difficult. Deep learning can be responsibly leveraged to improve productivity in addressing this task. The availability of data, however, remains a considerable obstacle. Hence, in this work, we introduce the Video ASD dataset--a dataset that contains video frame convolutional and attention map feature data--to foster further progress in the task of ASD classification. The original videos showcase children reacting to chemo-sensory stimuli, among auditory, touch, and vision This dataset contains the features of the frames spanning 2,467 videos, for a total of approximately 1.4 million frames. Additionally, head pose angles are included to account for head movement noise, as well as full-sentence text labels for the taste and smell videos that describe how the facial expression changes before, immediately after, and long after interaction with the stimuli. In addition to providing features, we also test foundation models on this data to showcase how movement noise affects performance and the need for more data and more complex labels.

Read more

9/10/2024

Ensemble Modeling of Multiple Physical Indicators to Dynamically Phenotype Autism Spectrum Disorder
Total Score

0

Ensemble Modeling of Multiple Physical Indicators to Dynamically Phenotype Autism Spectrum Disorder

Marie Huynh (Stanford University), Aaron Kline (Stanford University), Saimourya Surabhi (Stanford University), Kaitlyn Dunlap (Stanford University), Onur Cezmi Mutlu (Stanford University), Mohammadmahdi Honarmand (Stanford University), Parnian Azizian (Stanford University), Peter Washington (University of Hawaii at Manoa), Dennis P. Wall (Stanford University)

Early detection of autism, a neurodevelopmental disorder marked by social communication challenges, is crucial for timely intervention. Recent advancements have utilized naturalistic home videos captured via the mobile application GuessWhat. Through interactive games played between children and their guardians, GuessWhat has amassed over 3,000 structured videos from 382 children, both diagnosed with and without Autism Spectrum Disorder (ASD). This collection provides a robust dataset for training computer vision models to detect ASD-related phenotypic markers, including variations in emotional expression, eye contact, and head movements. We have developed a protocol to curate high-quality videos from this dataset, forming a comprehensive training set. Utilizing this set, we trained individual LSTM-based models using eye gaze, head positions, and facial landmarks as input features, achieving test AUCs of 86%, 67%, and 78%, respectively. To boost diagnostic accuracy, we applied late fusion techniques to create ensemble models, improving the overall AUC to 90%. This approach also yielded more equitable results across different genders and age groups. Our methodology offers a significant step forward in the early detection of ASD by potentially reducing the reliance on subjective assessments and making early identification more accessibly and equitable.

Read more

8/26/2024

🎯

Total Score

0

MMASD+: A Novel Dataset for Privacy-Preserving Behavior Analysis of Children with Autism Spectrum Disorder

Pavan Uttej Ravva, Behdokht Kiafar, Pinar Kullu, Jicheng Li, Anjana Bhat, Roghayeh Leila Barmaki

Autism spectrum disorder (ASD) is characterized by significant challenges in social interaction and comprehending communication signals. Recently, therapeutic interventions for ASD have increasingly utilized Deep learning powered-computer vision techniques to monitor individual progress over time. These models are trained on private, non-public datasets from the autism community, creating challenges in comparing results across different models due to privacy-preserving data-sharing issues. This work introduces MMASD+, an enhanced version of the novel open-source dataset called Multimodal ASD (MMASD). MMASD+ consists of diverse data modalities, including 3D-Skeleton, 3D Body Mesh, and Optical Flow data. It integrates the capabilities of Yolov8 and Deep SORT algorithms to distinguish between the therapist and children, addressing a significant barrier in the original dataset. Additionally, a Multimodal Transformer framework is proposed to predict 11 action types and the presence of ASD. This framework achieves an accuracy of 95.03% for predicting action types and 96.42% for predicting ASD presence, demonstrating over a 10% improvement compared to models trained on single data modalities. These findings highlight the advantages of integrating multiple data modalities within the Multimodal Transformer framework.

Read more

8/30/2024

Weakly-supervised Autism Severity Assessment in Long Videos
Total Score

0

Weakly-supervised Autism Severity Assessment in Long Videos

Abid Ali, Mahmoud Ali, Jean-Marc Odobez, Camilla Barbini, S'everine Dubuisson, Francois Bremond, Susanne Thummler

Autism Spectrum Disorder (ASD) is a diverse collection of neurobiological conditions marked by challenges in social communication and reciprocal interactions, as well as repetitive and stereotypical behaviors. Atypical behavior patterns in a long, untrimmed video can serve as biomarkers for children with ASD. In this paper, we propose a video-based weakly-supervised method that takes spatio-temporal features of long videos to learn typical and atypical behaviors for autism detection. On top of that, we propose a shallow TCN-MLP network, which is designed to further categorize the severity score. We evaluate our method on actual evaluation videos of children with autism collected and annotated (for severity score) by clinical professionals. Experimental results demonstrate the effectiveness of behavioral biomarkers that could help clinicians in autism spectrum analysis.

Read more

7/15/2024