Terrain-Aware Stride-Level Trajectory Forecasting for a Powered Hip Exoskeleton via Vision and Kinematics Fusion
2404.11945
0
0
Abstract
Powered hip exoskeletons have shown the ability for locomotion assistance during treadmill walking. However, providing suitable assistance in real-world walking scenarios which involve changing terrain remains challenging. Recent research suggests that forecasting the lower limb joint's angles could provide target trajectories for exoskeletons and prostheses, and the performance could be improved with visual information. In this letter, We share a real-world dataset of 10 healthy subjects walking through five common types of terrain with stride-level label. We design a network called Sandwich Fusion Transformer for Image and Kinematics (SFTIK), which predicts the thigh angle of the ensuing stride given the terrain images at the beginning of the preceding and the ensuing stride and the IMU time series during the preceding stride. We introduce width-level patchify, tailored for egocentric terrain images, to reduce the computational demands. We demonstrate the proposed sandwich input and fusion mechanism could significantly improve the forecasting performance. Overall, the SFTIK outperforms baseline methods, achieving a computational efficiency of 3.31 G Flops, and root mean square error (RMSE) of 3.445 textpm 0.804textdegree and Pearson's correlation coefficient (PCC) of 0.971 textpm 0.025. The results demonstrate that SFTIK could forecast the thigh's angle accurately with low computational cost, which could serve as a terrain adaptive trajectory planning method for hip exoskeletons. Codes and data are available at https://github.com/RuoqiZhao116/SFTIK.
Create account to get full access
Overview
- This paper presents a method for terrain-aware stride-level trajectory forecasting for a powered hip exoskeleton using a fusion of vision and kinematics data.
- The goal is to improve the performance and safety of exoskeleton-assisted locomotion on uneven terrain by predicting the user's intended foot placement and adjusting the exoskeleton's behavior accordingly.
- The proposed approach combines visual perception of the terrain with kinematic measurements from the exoskeleton to generate accurate stride-level trajectory forecasts.
Plain English Explanation
Powered exoskeletons are wearable robotic devices that can assist and enhance human movement, particularly for individuals with mobility challenges. However, navigating uneven or unpredictable terrain can be a significant challenge for these systems. This paper introduces a new method to address this problem.
The key idea is to combine two sources of information - vision and kinematics - to better predict where the user's foot will land on the ground during each step. By using a camera to "see" the upcoming terrain, and sensors on the exoskeleton to track the user's movements, the system can anticipate the user's intended foot placement and adjust the exoskeleton's behavior accordingly.
This "terrain-aware" approach allows the exoskeleton to provide more seamless and robust assistance, adapting its movements to match the user's intended path. This can improve the overall performance, safety, and comfort of exoskeleton-assisted locomotion, especially in challenging outdoor environments with uneven ground.
The authors demonstrate the effectiveness of their method through experiments and show that it outperforms approaches that rely on vision or kinematics alone. By fusing these two complementary data sources, the system can make more accurate stride-level trajectory forecasts, leading to better exoskeleton control and a more natural, intuitive user experience.
Technical Explanation
The proposed method uses a combination of visual perception and kinematic measurements to generate accurate stride-level trajectory forecasts for a powered hip exoskeleton.
The visual perception component uses a depth camera to capture a 3D point cloud representation of the upcoming terrain. This terrain information is then processed using a neural network to extract relevant features, such as the local terrain geometry and obstacle locations.
The kinematic component measures the user's joint angles, limb positions, and other motion parameters using sensors embedded in the exoskeleton. These kinematic measurements are used to infer the user's current state and intended movement patterns.
The vision and kinematic data are then fused using a recurrent neural network architecture that learns to predict the user's intended foot placement over the next few strides. This stride-level trajectory forecast is then used to adapt the exoskeleton's control strategy, enabling it to provide more seamless and responsive assistance as the user navigates uneven terrain.
The authors evaluate their approach through experiments on a custom-built powered hip exoskeleton platform. The results demonstrate that the proposed vision-kinematics fusion method outperforms approaches that rely on either vision or kinematics alone, leading to more accurate trajectory forecasts and improved exoskeleton performance.
Critical Analysis
The authors have addressed an important challenge in the field of exoskeleton-assisted locomotion – the need for robust and adaptive control strategies that can handle uneven terrain. By combining visual perception and kinematic data, their approach represents a promising step forward in this area.
One potential limitation of the study is the reliance on a custom-built exoskeleton platform, which may limit the generalizability of the findings. It would be valuable to see the method applied and evaluated on a wider range of exoskeleton systems to better understand its robustness and versatility.
Additionally, the authors acknowledge that their current method assumes a known relationship between the user's kinematic state and their intended foot placement. In more complex or unpredictable scenarios, this assumption may not hold, and the system's performance could degrade. Further research into more advanced user intent recognition and prediction models could help address this limitation.
Another area for potential improvement is the integration of the vision-kinematics fusion approach with higher-level planning and decision-making algorithms. By considering the broader context of the user's locomotion goals and environmental constraints, the exoskeleton's control strategy could potentially be further optimized for safety, efficiency, and user experience.
Conclusion
This paper presents a novel approach to terrain-aware stride-level trajectory forecasting for powered hip exoskeletons. By fusing visual perception of the upcoming terrain with kinematic measurements of the user's movements, the proposed method can generate accurate predictions of the user's intended foot placement, enabling the exoskeleton to provide more responsive and adaptive assistance.
The authors' experimental results demonstrate the benefits of this vision-kinematics fusion approach, showing improved performance compared to using vision or kinematics alone. This work represents an important step forward in enhancing the capabilities of exoskeleton-assisted locomotion, particularly in challenging outdoor environments with uneven terrain.
As the field of powered exoskeletons continues to evolve, the techniques and insights presented in this paper could have broader implications for the design of more intelligent, adaptive, and user-centric assistive robotic systems. Further research and development in this area could lead to significant improvements in the quality of life for individuals with mobility challenges.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
⚙️
Enhancing Prosthetic Safety and Environmental Adaptability: A Visual-Inertial Prosthesis Motion Estimation Approach on Uneven Terrains
Chuheng Chen, Xinxing Chen, Shucong Yin, Yuxuan Wang, Binxin Huang, Yuquan Leng, Chenglong Fu
0
0
Environment awareness is crucial for enhancing walking safety and stability of amputee wearing powered prosthesis when crossing uneven terrains such as stairs and obstacles. However, existing environmental perception systems for prosthesis only provide terrain types and corresponding parameters, which fails to prevent potential collisions when crossing uneven terrains and may lead to falls and other severe consequences. In this paper, a visual-inertial motion estimation approach is proposed for prosthesis to perceive its movement and the changes of spatial relationship between the prosthesis and uneven terrain when traversing them. To achieve this, we estimate the knee motion by utilizing a depth camera to perceive the environment and align feature points extracted from stairs and obstacles. Subsequently, an error-state Kalman filter is incorporated to fuse the inertial data into visual estimations to reduce the feature extraction error and obtain a more robust estimation. The motion of prosthetic joint and toe are derived using the prosthesis model parameters. Experiment conducted on our collected dataset and stair walking trials with a powered prosthesis shows that the proposed method can accurately tracking the motion of the human leg and prosthesis with an average root-mean-square error of toe trajectory less than 5 cm. The proposed method is expected to enable the environmental adaptive control for prosthesis, thereby enhancing amputee's safety and mobility in uneven terrains.
4/30/2024
Bi-level Trajectory Optimization on Uneven Terrains with Differentiable Wheel-Terrain Interaction Model
Amith Manoharan, Aditya Sharma, Himani Belsare, Kaustab Pal, K. Madhava Krishna, Arun Kumar Singh
0
0
Navigation of wheeled vehicles on uneven terrain necessitates going beyond the 2D approaches for trajectory planning. Specifically, it is essential to incorporate the full 6dof variation of vehicle pose and its associated stability cost in the planning process. To this end, most recent works aim to learn a neural network model to predict the vehicle evolution. However, such approaches are data-intensive and fraught with generalization issues. In this paper, we present a purely model-based approach that just requires the digital elevation information of the terrain. Specifically, we express the wheel-terrain interaction and 6dof pose prediction as a non-linear least squares (NLS) problem. As a result, trajectory planning can be viewed as a bi-level optimization. The inner optimization layer predicts the pose on the terrain along a given trajectory, while the outer layer deforms the trajectory itself to reduce the stability and kinematic costs of the pose. We improve the state-of-the-art in the following respects. First, we show that our NLS based pose prediction closely matches the output from a high-fidelity physics engine. This result coupled with the fact that we can query gradients of the NLS solver, makes our pose predictor, a differentiable wheel-terrain interaction model. We further leverage this differentiability to efficiently solve the proposed bi-level trajectory optimization problem. Finally, we perform extensive experiments, and comparison with a baseline to showcase the effectiveness of our approach in obtaining smooth, stable trajectories.
4/12/2024
Beyond Gait: Learning Knee Angle for Seamless Prosthesis Control in Multiple Scenarios
Pengwei Wang, Yilong Chen, Wan Su, Jie Wang, Teng Ma, Haoyong Yu
0
0
Deep learning models have become a powerful tool in knee angle estimation for lower limb prostheses, owing to their adaptability across various gait phases and locomotion modes. Current methods utilize Multi-Layer Perceptrons (MLP), Long-Short Term Memory Networks (LSTM), and Convolutional Neural Networks (CNN), predominantly analyzing motion information from the thigh. Contrary to these approaches, our study introduces a holistic perspective by integrating whole-body movements as inputs. We propose a transformer-based probabilistic framework, termed the Angle Estimation Probabilistic Model (AEPM), that offers precise angle estimations across extensive scenarios beyond walking. AEPM achieves an overall RMSE of 6.70 degrees, with an RMSE of 3.45 degrees in walking scenarios. Compared to the state of the art, AEPM has improved the prediction accuracy for walking by 11.31%. Our method can achieve seamless adaptation between different locomotion modes. Also, this model can be utilized to analyze the synergy between the knee and other joints. We reveal that the whole body movement has valuable information for knee movement, which can provide insights into designing sensors for prostheses. The code is available at https://github.com/penway/Beyond-Gait-AEPM.
4/11/2024
🌐
TOP-Nav: Legged Navigation Integrating Terrain, Obstacle and Proprioception Estimation
Junli Ren, Yikai Liu, Yingru Dai, Guijin Wang
0
0
Legged navigation is typically examined within open-world, off-road, and challenging environments. In these scenarios, estimating external disturbances requires a complex synthesis of multi-modal information. This underlines a major limitation in existing works that primarily focus on avoiding obstacles. In this work, we propose TOP-Nav, a novel legged navigation framework that integrates a comprehensive path planner with Terrain awareness, Obstacle avoidance and close-loop Proprioception. TOP-Nav underscores the synergies between vision and proprioception in both path and motion planning. Within the path planner, we present and integrate a terrain estimator that enables the robot to select waypoints on terrains with higher traversability while effectively avoiding obstacles. In the motion planning level, we not only implement a locomotion controller to track the navigation commands, but also construct a proprioception advisor to provide motion evaluations for the path planner. Based on the close-loop motion feedback, we make online corrections for the vision-based terrain and obstacle estimations. Consequently, TOP-Nav achieves open-world navigation that the robot can handle terrains or disturbances beyond the distribution of prior knowledge and overcomes constraints imposed by visual conditions. Building upon extensive experiments conducted in both simulation and real-world environments, TOP-Nav demonstrates superior performance in open-world navigation compared to existing methods.
4/24/2024