BaboonLand Dataset: Tracking Primates in the Wild and Automating Behaviour Recognition from Drone Videos

Read original: arXiv:2405.17698 - Published 6/5/2024 by Isla Duporge, Maksim Kholiavchenko, Roi Harel, Scott Wolf, Dan Rubenstein, Meg Crofoot, Tanya Berger-Wolf, Stephen Lee, Julie Barreau, Jenna Kline and 2 others
Total Score

0

BaboonLand Dataset: Tracking Primates in the Wild and Automating Behaviour Recognition from Drone Videos

Sign in to get full access

or

If you already have an account, we'll log you in

Overview

  • This paper introduces the BaboonLand dataset, a new resource for tracking and recognizing behaviors of primates in the wild using drone footage.
  • The dataset includes over 1,000 hours of high-quality video of baboons in their natural habitat, along with detailed annotations of the animals' behaviors.
  • The researchers also present new computer vision models that can automatically detect and classify the behaviors of baboons in the video, enabling efficient and scalable monitoring of primate populations.

Plain English Explanation

The BaboonLand dataset is a valuable new resource for studying primate behavior in their natural environments. Researchers have collected over 1,000 hours of high-quality drone footage of baboons living in the wild, and they've carefully annotated the videos to label the different behaviors the animals exhibit, such as feeding, resting, socializing, and so on.

This dataset can be used to train computer vision models that can automatically detect and classify the behaviors of baboons in new video footage. This is a big advance, as manually watching and coding primate behavior is a very labor-intensive and time-consuming process. With these AI models, researchers can efficiently monitor primate populations and study their behaviors at a much larger scale than was previously possible.

The availability of the BaboonLand dataset and the associated behavior recognition models could have important implications for primate conservation efforts and our broader understanding of primate ecology and social dynamics. By making it easier to collect detailed behavioral data on primates in the wild, this research could lead to new insights that help us better protect these endangered species and their habitats.

Technical Explanation

The BaboonLand dataset contains over 1,000 hours of high-resolution drone footage of a troop of chacma baboons living in their natural habitat. The researchers used custom-built drones equipped with high-quality cameras to capture the videos, ensuring consistent and reliable data collection.

The video footage was then manually annotated by experts, who labeled each frame with the specific behaviors exhibited by the baboons, such as feeding, resting, grooming, etc. This resulted in a comprehensive "ethogram" - a detailed catalog of the various behaviors that can be observed in this primate population.

Building on this rich dataset, the researchers developed new computer vision models that can automatically detect and classify the baboons' behaviors in the video footage. These models were trained using a combination of supervised learning on the annotated data, as well as unsupervised learning techniques to capture more nuanced behavioral patterns.

The resulting behavior recognition system achieved high accuracy in identifying the baboons' actions, demonstrating the potential for this approach to enable efficient, large-scale monitoring of primate populations in the wild. This could be particularly valuable for long-term ecological studies and conservation efforts focused on endangered primate species.

Critical Analysis

The BaboonLand dataset and the associated behavior recognition models represent an impressive technical achievement, with the potential to significantly advance primate research and conservation. However, the researchers acknowledge several key limitations and areas for future work.

One limitation is that the dataset is currently focused on a single baboon troop, and it's unclear how well the behavior recognition models would generalize to other primate populations or species. Further validation and testing on more diverse datasets would be needed to assess the broader applicability of this approach.

Additionally, while the automated behavior recognition system achieved high accuracy, there may still be some subtle behaviors or contextual nuances that are difficult for the models to reliably detect. Continued refinement of the models and incorporation of additional data sources (e.g., audio, physiological sensors) could help address these challenges.

It's also important to consider the potential ethical and privacy implications of using drone-based monitoring of wild animals. The researchers note that they took steps to minimize disturbance to the baboons and obtain appropriate permissions, but further work may be needed to ensure the welfare of the primates and address any concerns from local communities or other stakeholders.

Conclusion

The BaboonLand dataset and the associated behavior recognition models represent a significant advance in the field of primate research and conservation. By enabling efficient, large-scale monitoring of primate populations in their natural habitats, this work could lead to new insights into primate behavior, ecology, and social dynamics, as well as more effective strategies for protecting these endangered species and their ecosystems.

While the current research has some limitations, the overall approach demonstrates the potential of combining drone technology, computer vision, and rich behavioral datasets to transform the way we study and conserve wild animal populations. As this field continues to evolve, we can expect to see even more innovative applications of these tools to support our understanding and stewardship of the natural world.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

BaboonLand Dataset: Tracking Primates in the Wild and Automating Behaviour Recognition from Drone Videos
Total Score

0

BaboonLand Dataset: Tracking Primates in the Wild and Automating Behaviour Recognition from Drone Videos

Isla Duporge, Maksim Kholiavchenko, Roi Harel, Scott Wolf, Dan Rubenstein, Meg Crofoot, Tanya Berger-Wolf, Stephen Lee, Julie Barreau, Jenna Kline, Michelle Ramirez, Charles Stewart

Using drones to track multiple individuals simultaneously in their natural environment is a powerful approach for better understanding group primate behavior. Previous studies have demonstrated that it is possible to automate the classification of primate behavior from video data, but these studies have been carried out in captivity or from ground-based cameras. To understand group behavior and the self-organization of a collective, the whole troop needs to be seen at a scale where behavior can be seen in relation to the natural environment in which ecological decisions are made. This study presents a novel dataset from drone videos for baboon detection, tracking, and behavior recognition. The baboon detection dataset was created by manually annotating all baboons in drone videos with bounding boxes. A tiling method was subsequently applied to create a pyramid of images at various scales from the original 5.3K resolution images, resulting in approximately 30K images used for baboon detection. The tracking dataset is derived from the detection dataset, where all bounding boxes are assigned the same ID throughout the video. This process resulted in half an hour of very dense tracking data. The behavior recognition dataset was generated by converting tracks into mini-scenes, a video subregion centered on each animal; each mini-scene was manually annotated with 12 distinct behavior types, resulting in over 20 hours of data. Benchmark results show mean average precision (mAP) of 92.62% for the YOLOv8-X detection model, multiple object tracking precision (MOTA) of 63.81% for the BotSort tracking algorithm, and micro top-1 accuracy of 63.97% for the X3D behavior recognition model. Using deep learning to classify wildlife behavior from drone footage facilitates non-invasive insight into the collective behavior of an entire group.

Read more

6/5/2024

From Forest to Zoo: Great Ape Behavior Recognition with ChimpBehave
Total Score

0

From Forest to Zoo: Great Ape Behavior Recognition with ChimpBehave

Michael Fuchs, Emilie Genty, Adrian Bangerter, Klaus Zuberbuhler, Paul Cotofrei

This paper addresses the significant challenge of recognizing behaviors in non-human primates, specifically focusing on chimpanzees. Automated behavior recognition is crucial for both conservation efforts and the advancement of behavioral research. However, it is significantly hindered by the labor-intensive process of manual video annotation. Despite the availability of large-scale animal behavior datasets, the effective application of machine learning models across varied environmental settings poses a critical challenge, primarily due to the variability in data collection contexts and the specificity of annotations. In this paper, we introduce ChimpBehave, a novel dataset featuring over 2 hours of video (approximately 193,000 video frames) of zoo-housed chimpanzees, meticulously annotated with bounding boxes and behavior labels for action recognition. ChimpBehave uniquely aligns its behavior classes with existing datasets, allowing for the study of domain adaptation and cross-dataset generalization methods between different visual settings. Furthermore, we benchmark our dataset using a state-of-the-art CNN-based action recognition model, providing the first baseline results for both within and cross-dataset settings. The dataset, models, and code can be accessed at: https://github.com/MitchFuchs/ChimpBehave

Read more

5/31/2024

👀

Total Score

0

Computer Vision for Primate Behavior Analysis in the Wild

Richard Vogg, Timo Luddecke, Jonathan Henrich, Sharmita Dey, Matthias Nuske, Valentin Hassler, Derek Murphy, Julia Fischer, Julia Ostner, Oliver Schulke, Peter M. Kappeler, Claudia Fichtel, Alexander Gail, Stefan Treue, Hansjorg Scherberger, Florentin Worgotter, Alexander S. Ecker

Advances in computer vision as well as increasingly widespread video-based behavioral monitoring have great potential for transforming how we study animal cognition and behavior. However, there is still a fairly large gap between the exciting prospects and what can actually be achieved in practice today, especially in videos from the wild. With this perspective paper, we want to contribute towards closing this gap, by guiding behavioral scientists in what can be expected from current methods and steering computer vision researchers towards problems that are relevant to advance research in animal behavior. We start with a survey of the state-of-the-art methods for computer vision problems that are directly relevant to the video-based study of animal behavior, including object detection, multi-individual tracking, individual identification, and (inter)action recognition. We then review methods for effort-efficient learning, which is one of the biggest challenges from a practical perspective. Finally, we close with an outlook into the future of the emerging field of computer vision for animal behavior, where we argue that the field should develop approaches to unify detection, tracking, identification and (inter)action recognition in a single, video-based framework.

Read more

8/13/2024

Watching Swarm Dynamics from Above: A Framework for Advanced Object Tracking in Drone Videos
Total Score

0

Watching Swarm Dynamics from Above: A Framework for Advanced Object Tracking in Drone Videos

Duc Pham, Matthew Hansen, F'elicie Dhellemmens, Jens Krause, Pia Bideau

Easily accessible sensors, like drones with diverse onboard sensors, have greatly expanded studying animal behavior in natural environments. Yet, analyzing vast, unlabeled video data, often spanning hours, remains a challenge for machine learning, especially in computer vision. Existing approaches often analyze only a few frames. Our focus is on long-term animal behavior analysis. To address this challenge, we utilize classical probabilistic methods for state estimation, such as particle filtering. By incorporating recent advancements in semantic object segmentation, we enable continuous tracking of rapidly evolving object formations, even in scenarios with limited data availability. Particle filters offer a provably optimal algorithmic structure for recursively adding new incoming information. We propose a novel approach for tracking schools of fish in the open ocean from drone videos. Our framework not only performs classical object tracking in 2D, instead it tracks the position and spatial expansion of the fish school in world coordinates by fusing video data and the drone's on board sensor information (GPS and IMU). The presented framework for the first time allows researchers to study collective behavior of fish schools in its natural social and environmental context in a non-invasive and scalable way.

Read more

6/13/2024