Towards Sustainable Personalized On-Device Human Activity Recognition with TinyML and Cloud-Enabled Auto Deployment

Read original: arXiv:2409.00093 - Published 9/4/2024 by Bidyut Saha, Riya Samanta, Soumya K Ghosh, Ram Babu Roy
Total Score

0

Towards Sustainable Personalized On-Device Human Activity Recognition with TinyML and Cloud-Enabled Auto Deployment

Sign in to get full access

or

If you already have an account, we'll log you in

Overview

  • Explores sustainable and personalized on-device human activity recognition using TinyML and cloud-enabled auto-deployment
  • Focuses on developing efficient models that can run on resource-constrained edge devices while maintaining high accuracy
  • Leverages cloud infrastructure for automated model training, optimization, and deployment to edge devices

Plain English Explanation

This research paper presents an approach to enable sustainable and personalized on-device human activity recognition using TinyML and cloud-enabled auto-deployment. The key idea is to develop efficient machine learning models that can run directly on resource-constrained edge devices, such as smartphones or wearables, while maintaining high accuracy in recognizing a person's activities.

The researchers leverage the cloud infrastructure to automate the process of training, optimizing, and deploying these compact models to the edge devices. This cloud-enabled approach allows for personalization, where the models can be tailored to individual users, and sustainable updates, where the models can be continuously improved and deployed without the need for manual intervention.

By combining the power of on-device TinyML with the scalability and automation of cloud-based deployment, the researchers aim to create a sustainable and personalized human activity recognition system that can be widely deployed and maintained over time.

Technical Explanation

The paper proposes a framework that integrates TinyML models for on-device inference and a cloud-based infrastructure for automated model training, optimization, and deployment. The key components of the framework include:

  1. Edge Device: The on-device TinyML models are designed to run efficiently on resource-constrained edge devices, such as smartphones or wearables, to perform human activity recognition tasks.

  2. Cloud Infrastructure: The cloud-based system is responsible for training personalized models for each user, optimizing the models for deployment on the edge devices, and automatically pushing the updated models to the corresponding devices.

  3. Automated Model Lifecycle: The cloud infrastructure automates the entire model lifecycle, from data collection and model training to deployment and updates, enabling a sustainable and personalized human activity recognition system.

The researchers evaluate their framework using various datasets and benchmark the performance of the TinyML models against larger, more complex models. They also demonstrate the effectiveness of the cloud-enabled auto-deployment approach in maintaining personalized models on edge devices over time.

Critical Analysis

The research presented in this paper addresses an important challenge in the field of human activity recognition - enabling efficient and personalized on-device inference while leveraging cloud-based automation for model updates and deployment. The authors have made a compelling case for the need to balance on-device performance with cloud-enabled scalability and personalization.

One potential limitation of the study is the scope of the datasets and use cases explored. The authors focus primarily on standard human activity recognition datasets, which may not fully capture the diversity of real-world scenarios and user preferences. Expanding the evaluation to include more varied datasets and real-world deployments could provide additional insights into the practical implications and challenges of the proposed framework.

Additionally, the paper does not delve deeply into the potential privacy and security concerns associated with the cloud-based infrastructure and the transfer of user data. As personalized models are trained and deployed, it will be important to address these aspects to ensure the framework is truly sustainable and trustworthy from the user's perspective.

Further research could also explore the integration of sensor data augmentation techniques to enhance the robustness and generalization of the TinyML models, as well as investigate the potential of federated learning approaches to distribute the model training and update process while preserving user privacy.

Conclusion

This research paper presents an innovative approach to enable sustainable and personalized on-device human activity recognition by leveraging the strengths of TinyML and cloud-enabled auto-deployment. The proposed framework addresses the need for efficient models that can run on resource-constrained edge devices while maintaining high accuracy, and it automates the model lifecycle to ensure continuous personalization and updates.

The key contribution of this work lies in the integration of on-device TinyML and cloud-based infrastructure, which enables a scalable and sustainable solution for human activity recognition. As edge devices become more prevalent and user preferences become increasingly diverse, this type of approach can have significant implications for the development of personalized, privacy-preserving, and continuously improving applications in the domain of human activity recognition and beyond.



This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!

Follow @aimodelsfyi on 𝕏 →

Related Papers

Towards Sustainable Personalized On-Device Human Activity Recognition with TinyML and Cloud-Enabled Auto Deployment
Total Score

0

Towards Sustainable Personalized On-Device Human Activity Recognition with TinyML and Cloud-Enabled Auto Deployment

Bidyut Saha, Riya Samanta, Soumya K Ghosh, Ram Babu Roy

Human activity recognition (HAR) holds immense potential for transforming health and fitness monitoring, yet challenges persist in achieving personalized outcomes and sustainability for on-device continuous inferences. This work introduces a wrist-worn smart band designed to address these challenges through a novel combination of on-device TinyML-driven computing and cloud-enabled auto-deployment. Leveraging inertial measurement unit (IMU) sensors and a customized 1D Convolutional Neural Network (CNN) for personalized HAR, users can tailor activity classes to their unique movement styles with minimal calibration. By utilising TinyML for local computations, the smart band reduces the necessity for constant data transmission and radio communication, which in turn lowers power consumption and reduces carbon footprint. This method also enhances the privacy and security of user data by limiting its transmission. Through transfer learning and fine-tuning on user-specific data, the system achieves a 37% increase in accuracy over generalized models in personalized settings. Evaluation using three benchmark datasets, WISDM, PAMAP2, and the BandX demonstrates its effectiveness across various activity domains. Additionally, this work presents a cloud-supported framework for the automatic deployment of TinyML models to remote wearables, enabling seamless customization and on-device inference, even with limited target data. By combining personalized HAR with sustainable strategies for on-device continuous inferences, this system represents a promising step towards fostering healthier and more sustainable societies worldwide.

Read more

9/4/2024

On-Device Training Empowered Transfer Learning For Human Activity Recognition
Total Score

0

On-Device Training Empowered Transfer Learning For Human Activity Recognition

Pixi Kang, Julian Moosmann, Sizhen Bian, Michele Magno

Human Activity Recognition (HAR) is an attractive topic to perceive human behavior and supplying assistive services. Besides the classical inertial unit and vision-based HAR methods, new sensing technologies, such as ultrasound and body-area electric fields, have emerged in HAR to enhance user experience and accommodate new application scenarios. As those sensors are often paired with AI for HAR, they frequently encounter challenges due to limited training data compared to the more widely IMU or vision-based HAR solutions. Additionally, user-induced concept drift (UICD) is common in such HAR scenarios. UICD is characterized by deviations in the sample distribution of new users from that of the training participants, leading to deteriorated recognition performance. This paper proposes an on-device transfer learning (ODTL) scheme tailored for energy- and resource-constrained IoT edge devices. Optimized on-device training engines are developed for two representative MCU-level edge computing platforms: STM32F756ZG and GAP9. Based on this, we evaluated the ODTL benefits in three HAR scenarios: body capacitance-based gym activity recognition, QVAR- and ultrasonic-based hand gesture recognition. We demonstrated an improvement of 3.73%, 17.38%, and 3.70% in the activity recognition accuracy, respectively. Besides this, we observed that the RISC-V-based GAP9 achieves 20x and 280x less latency and power consumption than STM32F7 MCU during the ODTL deployment, demonstrating the advantages of employing the latest low-power parallel computing devices for edge tasks.

Read more

7/8/2024

Millimeter Wave Radar-based Human Activity Recognition for Healthcare Monitoring Robot
Total Score

0

Millimeter Wave Radar-based Human Activity Recognition for Healthcare Monitoring Robot

Zhanzhong Gu, Xiangjian He, Gengfa Fang, Chengpei Xu, Feng Xia, Wenjing Jia

Healthcare monitoring is crucial, especially for the daily care of elderly individuals living alone. It can detect dangerous occurrences, such as falls, and provide timely alerts to save lives. Non-invasive millimeter wave (mmWave) radar-based healthcare monitoring systems using advanced human activity recognition (HAR) models have recently gained significant attention. However, they encounter challenges in handling sparse point clouds, achieving real-time continuous classification, and coping with limited monitoring ranges when statically mounted. To overcome these limitations, we propose RobHAR, a movable robot-mounted mmWave radar system with lightweight deep neural networks for real-time monitoring of human activities. Specifically, we first propose a sparse point cloud-based global embedding to learn the features of point clouds using the light-PointNet (LPN) backbone. Then, we learn the temporal pattern with a bidirectional lightweight LSTM model (BiLiLSTM). In addition, we implement a transition optimization strategy, integrating the Hidden Markov Model (HMM) with Connectionist Temporal Classification (CTC) to improve the accuracy and robustness of the continuous HAR. Our experiments on three datasets indicate that our method significantly outperforms the previous studies in both discrete and continuous HAR tasks. Finally, we deploy our system on a movable robot-mounted edge computing platform, achieving flexible healthcare monitoring in real-world scenarios.

Read more

5/6/2024

🤷

Total Score

0

Unsupervised Statistical Feature-Guided Diffusion Model for Sensor-based Human Activity Recognition

Si Zuo, Vitor Fortes Rey, Sungho Suh, Stephan Sigg, Paul Lukowicz

Human activity recognition (HAR) from on-body sensors is a core functionality in many AI applications: from personal health, through sports and wellness to Industry 4.0. A key problem holding up progress in wearable sensor-based HAR, compared to other ML areas, such as computer vision, is the unavailability of diverse and labeled training data. Particularly, while there are innumerable annotated images available in online repositories, freely available sensor data is sparse and mostly unlabeled. We propose an unsupervised statistical feature-guided diffusion model specifically optimized for wearable sensor-based human activity recognition with devices such as inertial measurement unit (IMU) sensors. The method generates synthetic labeled time-series sensor data without relying on annotated training data. Thereby, it addresses the scarcity and annotation difficulties associated with real-world sensor data. By conditioning the diffusion model on statistical information such as mean, standard deviation, Z-score, and skewness, we generate diverse and representative synthetic sensor data. We conducted experiments on public human activity recognition datasets and compared the method to conventional oversampling and state-of-the-art generative adversarial network methods. Experimental results demonstrate that this can improve the performance of human activity recognition and outperform existing techniques.

Read more

5/21/2024