Non-stationary BERT: Exploring Augmented IMU Data For Robust Human Activity Recognition
0
Sign in to get full access
Overview
- Explores using inertial measurement units (IMUs) and a pre-trained BERT model for human activity recognition (HAR)
- Proposes a lightweight AI system that can run on edge devices
- Evaluates the approach on a challenging HAR dataset
Plain English Explanation
This paper investigates using inertial measurement units (IMUs) and a pre-trained BERT model for the task of human activity recognition (HAR). The key idea is to leverage the rich feature representations learned by BERT on natural language data and apply them to sensor data for HAR.
The researchers develop a lightweight AI system that can run efficiently on edge devices like smartphones or wearables. This is important because HAR systems need to operate in real-time and on-device to be useful for applications like activity tracking, health monitoring, and smart home control.
The system is evaluated on a challenging dataset that includes a diverse set of daily activities. The results show that the approach can achieve competitive performance compared to more complex models, while being much more efficient and able to run on low-power devices.
Technical Explanation
The paper proposes a light-weight AI system for human activity recognition (HAR) that utilizes inertial measurement unit (IMU) data and a pre-trained BERT model.
The key components are:
-
IMU Data Preprocessing: The raw IMU data (accelerometer and gyroscope) is preprocessed by segmenting it into fixed-length windows and applying standard normalization techniques.
-
BERT Feature Extraction: The preprocessed IMU data is passed through a pre-trained BERT model to extract rich feature representations. BERT was originally designed for natural language processing, but the authors hypothesize that its general-purpose feature learning capabilities can also be effective for sensor data.
-
Lightweight Classification Head: A small, fully-connected neural network is trained on top of the BERT features to perform the final activity classification. This lightweight design allows the system to run efficiently on edge devices.
The approach is evaluated on a challenging HAR dataset that includes a diverse set of daily activities. The results demonstrate that the proposed system can achieve competitive performance compared to more complex models, while being significantly more efficient and able to run on low-power hardware.
Critical Analysis
The paper presents a compelling approach to human activity recognition (HAR) that leverages the power of pre-trained BERT models in a lightweight, edge-deployable system.
One potential limitation is the reliance on a fixed-length window approach for preprocessing the IMU data. This may not be ideal for capturing the full temporal dynamics of some activities. Exploring more advanced sensor data augmentation techniques could be an area for further research.
Additionally, the authors do not provide much insight into the specific feature representations learned by BERT and how they relate to the underlying sensor data characteristics. A deeper analysis of the learned features could help shed light on the model's inner workings and potentially lead to further improvements.
Overall, the paper presents a promising direction for lightweight AI solutions in the HAR domain, with potential applications in areas like activity tracking, health monitoring, and smart home control.
Conclusion
This paper explores the use of inertial measurement units (IMUs) and a pre-trained BERT model for the task of human activity recognition (HAR). The key contribution is a lightweight AI system that can run efficiently on edge devices while achieving competitive performance on a challenging HAR dataset.
The approach leverages the rich feature representations learned by BERT on natural language data and applies them to sensor data, demonstrating the versatility of this powerful language model. The lightweight design of the classification head allows the system to be deployed on low-power hardware, making it a promising solution for real-time, on-device HAR applications.
Overall, this research presents an exciting step towards practical, edge-based HAR systems that can enable a wide range of smart and personalized services in our daily lives.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
Non-stationary BERT: Exploring Augmented IMU Data For Robust Human Activity Recognition
Ning Sun, Yufei Wang, Yuwei Zhang, Jixiang Wan, Shenyue Wang, Ping Liu, Xudong Zhang
Human Activity Recognition (HAR) has gained great attention from researchers due to the popularity of mobile devices and the need to observe users' daily activity data for better human-computer interaction. In this work, we collect a human activity recognition dataset called OPPOHAR consisting of phone IMU data. To facilitate the employment of HAR system in mobile phone and to achieve user-specific activity recognition, we propose a novel light-weight network called Non-stationary BERT with a two-stage training method. We also propose a simple yet effective data augmentation method to explore the deeper relationship between the accelerator and gyroscope data from the IMU. The network achieves the state-of-the-art performance testing on various activity recognition datasets and the data augmentation method demonstrates its wide applicability.
Read more9/26/2024
0
New!Language-centered Human Activity Recognition
Hua Yan, Heng Tan, Yi Ding, Pengfei Zhou, Vinod Namboodiri, Yu Yang
Human Activity Recognition (HAR) using Inertial Measurement Unit (IMU) sensors is critical for applications in healthcare, safety, and industrial production. However, variations in activity patterns, device types, and sensor placements create distribution gaps across datasets, reducing the performance of HAR models. To address this, we propose LanHAR, a novel system that leverages Large Language Models (LLMs) to generate semantic interpretations of sensor readings and activity labels for cross-dataset HAR. This approach not only mitigates cross-dataset heterogeneity but also enhances the recognition of new activities. LanHAR employs an iterative re-generation method to produce high-quality semantic interpretations with LLMs and a two-stage training framework that bridges the semantic interpretations of sensor readings and activity labels. This ultimately leads to a lightweight sensor encoder suitable for mobile deployment, enabling any sensor reading to be mapped into the semantic interpretation space. Experiments on four public datasets demonstrate that our approach significantly outperforms state-of-the-art methods in both cross-dataset HAR and new activity recognition. The source code will be made publicly available.
Read more10/4/2024
0
Sensor Data Augmentation from Skeleton Pose Sequences for Improving Human Activity Recognition
Parham Zolfaghari, Vitor Fortes Rey, Lala Ray, Hyun Kim, Sungho Suh, Paul Lukowicz
The proliferation of deep learning has significantly advanced various fields, yet Human Activity Recognition (HAR) has not fully capitalized on these developments, primarily due to the scarcity of labeled datasets. Despite the integration of advanced Inertial Measurement Units (IMUs) in ubiquitous wearable devices like smartwatches and fitness trackers, which offer self-labeled activity data from users, the volume of labeled data remains insufficient compared to domains where deep learning has achieved remarkable success. Addressing this gap, in this paper, we propose a novel approach to improve wearable sensor-based HAR by introducing a pose-to-sensor network model that generates sensor data directly from 3D skeleton pose sequences. our method simultaneously trains the pose-to-sensor network and a human activity classifier, optimizing both data reconstruction and activity recognition. Our contributions include the integration of simultaneous training, direct pose-to-sensor generation, and a comprehensive evaluation on the MM-Fit dataset. Experimental results demonstrate the superiority of our framework with significant performance improvements over baseline methods.
Read more6/26/2024
0
Decoding Human Activities: Analyzing Wearable Accelerometer and Gyroscope Data for Activity Recognition
Utsab Saha, Sawradip Saha, Tahmid Kabir, Shaikh Anowarul Fattah, Mohammad Saquib
A person's movement or relative positioning can be effectively captured by different types of sensors and corresponding sensor output can be utilized in various manipulative techniques for the classification of different human activities. This letter proposes an effective scheme for human activity recognition, which introduces two unique approaches within a multi-structural architecture, named FusionActNet. The first approach aims to capture the static and dynamic behavior of a particular action by using two dedicated residual networks and the second approach facilitates the final decision-making process by introducing a guidance module. A two-stage training process is designed where at the first stage, residual networks are pre-trained separately by using static (where the human body is immobile) and dynamic (involving movement of the human body) data. In the next stage, the guidance module along with the pre-trained static or dynamic models are used to train the given sensor data. Here the guidance module learns to emphasize the most relevant prediction vector obtained from the static or dynamic models, which helps to effectively classify different human activities. The proposed scheme is evaluated using two benchmark datasets and compared with state-of-the-art methods. The results clearly demonstrate that our method outperforms existing approaches in terms of accuracy, precision, recall, and F1 score, achieving 97.35% and 95.35% accuracy on the UCI HAR and Motion-Sense datasets, respectively which highlights both the effectiveness and stability of the proposed scheme.
Read more7/10/2024