Redefining Data Pairing for Motion Retargeting Leveraging a Human Body Prior
0
Sign in to get full access
Overview
- Researchers propose a new approach to motion retargeting that leverages a human body prior
- Their method redefines the data pairing process to improve the accuracy and realism of motion retargeting
- The approach is demonstrated through experiments on various motion retargeting tasks
Plain English Explanation
Motion retargeting is the process of transferring recorded human motion to a different character or robot. This can be useful for applications like animating digital characters or controlling robotic systems. However, traditional methods for motion retargeting can struggle to capture the nuances and natural qualities of human movement.
The researchers in this paper introduce a new way to approach motion retargeting that takes advantage of a human body prior. This means they leverage our existing knowledge about the structure and capabilities of the human body to guide the retargeting process. By redefining how motion data is paired between the human and target character, they are able to produce retargeted motions that are more faithful to the original human movements.
Through a series of experiments, the researchers demonstrate the effectiveness of their approach on different motion retargeting tasks. Their method is shown to generate more realistic and accurate results compared to previous techniques.
Technical Explanation
The key innovation in this paper is the redefinition of the data pairing process for motion retargeting. Traditionally, motion retargeting aligns joint positions or trajectories between the human performer and the target character. However, this can lead to distortions or unrealistic motions, especially when the bodies have different proportions or degrees of freedom.
Instead, the researchers propose pairing the motion data based on functional joint correspondences. This means they match up joints based on their roles and behaviors within the kinematic structure, rather than just their spatial locations. For example, the elbow joint of the human would be paired with the elbow joint of the target character, even if their positions don't perfectly align.
By leveraging this human body prior, the method is able to better preserve the natural qualities of the original human motion during the retargeting process. The researchers demonstrate the effectiveness of their approach through experiments on various motion retargeting tasks, showing improved accuracy and realism compared to prior techniques.
Critical Analysis
The researchers acknowledge some limitations of their approach. For instance, the method currently relies on a pre-defined set of functional joint correspondences, which may not generalize well to highly diverse or unique character designs. Additionally, the experiments focus on relatively simple motions, and the approach may face challenges with more complex or dynamic movements.
Further research could explore ways to automatically infer the functional joint correspondences, or to dynamically adjust the pairing during the retargeting process. Incorporating additional contextual information, such as the character's physical properties or the task being performed, could also help improve the generalization and robustness of the method.
Overall, the researchers present a promising new direction for motion retargeting that leverages our understanding of human biomechanics. By redefining the data pairing process, their approach represents an important step towards more natural and realistic motion transfer between different characters and platforms.
Conclusion
This paper introduces a novel motion retargeting method that redefines the data pairing process to better leverage a human body prior. By matching joints based on their functional roles rather than just spatial alignment, the researchers are able to produce retargeted motions that more faithfully capture the nuances and natural qualities of original human movements.
The demonstrated improvements in accuracy and realism suggest that this approach could have significant implications for a wide range of applications, from animating digital characters to controlling robotic systems. As the field of motion retargeting continues to evolve, techniques like the one presented in this paper will be crucial for bridging the gap between human and artificial motion in increasingly realistic and seamless ways.
This summary was produced with help from an AI and may contain inaccuracies - check out the links to read the original source documents!
Related Papers
0
Redefining Data Pairing for Motion Retargeting Leveraging a Human Body Prior
Xiyana Figuera, Soogeun Park, Hyemin Ahn
We propose MR HuBo(Motion Retargeting leveraging a HUman BOdy prior), a cost-effective and convenient method to collect high-quality upper body paired pose data, which is essential for data-driven motion retargeting methods. Unlike existing approaches which collect pose data by converting human MoCap poses into robot poses, our method goes in reverse. We first sample diverse random robot poses, and then convert them into human poses. However, since random robot poses can result in extreme and infeasible human poses, we propose an additional technique to sort out extreme poses by exploiting a human body prior trained from a large amount of human pose data. Our data collection method can be used for any humanoid robots, if one designs or optimizes the system's hyperparameters which include a size scale factor and the joint angle ranges for sampling. In addition to this data collection method, we also present a two-stage motion retargeting neural network that can be trained via supervised learning on a large amount of paired data. Compared to other learning-based methods trained via unsupervised learning, we found that our deep neural network trained with ample high-quality paired data achieved notable performance. Our experiments also show that our data filtering method yields better retargeting results than training the model with raw and noisy data. Our code and video results are available on https://sites.google.com/view/mr-hubo/
Read more10/2/2024
0
Unsupervised Neural Motion Retargeting for Humanoid Teleoperation
Satoshi Yagi, Mitsunori Tada, Eiji Uchibe, Suguru Kanoga, Takamitsu Matsubara, Jun Morimoto
This study proposes an approach to human-to-humanoid teleoperation using GAN-based online motion retargeting, which obviates the need for the construction of pairwise datasets to identify the relationship between the human and the humanoid kinematics. Consequently, it can be anticipated that our proposed teleoperation system will reduce the complexity and setup requirements typically associated with humanoid controllers, thereby facilitating the development of more accessible and intuitive teleoperation systems for users without robotics knowledge. The experiments demonstrated the efficacy of the proposed method in retargeting a range of upper-body human motions to humanoid, including a body jab motion and a basketball shoot motion. Moreover, the human-in-the-loop teleoperation performance was evaluated by measuring the end-effector position errors between the human and the retargeted humanoid motions. The results demonstrated that the error was comparable to those of conventional motion retargeting methods that require pairwise motion datasets. Finally, a box pick-and-place task was conducted to demonstrate the usability of the developed humanoid teleoperation system.
Read more6/4/2024
🤷
0
ImitationNet: Unsupervised Human-to-Robot Motion Retargeting via Shared Latent Space
Yashuai Yan, Esteve Valls Mascaro, Dongheui Lee
This paper introduces a novel deep-learning approach for human-to-robot motion retargeting, enabling robots to mimic human poses accurately. Contrary to prior deep-learning-based works, our method does not require paired human-to-robot data, which facilitates its translation to new robots. First, we construct a shared latent space between humans and robots via adaptive contrastive learning that takes advantage of a proposed cross-domain similarity metric between the human and robot poses. Additionally, we propose a consistency term to build a common latent space that captures the similarity of the poses with precision while allowing direct robot motion control from the latent space. For instance, we can generate in-between motion through simple linear interpolation between two projected human poses. We conduct a comprehensive evaluation of robot control from diverse modalities (i.e., texts, RGB videos, and key poses), which facilitates robot control for non-expert users. Our model outperforms existing works regarding human-to-robot retargeting in terms of efficiency and precision. Finally, we implemented our method in a real robot with self-collision avoidance through a whole-body controller to showcase the effectiveness of our approach. More information on our website https://evm7.github.io/UnsH2R/
Read more4/9/2024
0
A Robust Filter for Marker-less Multi-person Tracking in Human-Robot Interaction Scenarios
Enrico Martini, Harshil Parekh, Shaoting Peng, Nicola Bombieri, Nadia Figueroa
Pursuing natural and marker-less human-robot interaction (HRI) has been a long-standing robotics research focus, driven by the vision of seamless collaboration without physical markers. Marker-less approaches promise an improved user experience, but state-of-the-art struggles with the challenges posed by intrinsic errors in human pose estimation (HPE) and depth cameras. These errors can lead to issues such as robot jittering, which can significantly impact the trust users have in collaborative systems. We propose a filtering pipeline that refines incomplete 3D human poses from an HPE backbone and a single RGB-D camera to address these challenges, solving for occlusions that can degrade the interaction. Experimental results show that using the proposed filter leads to more consistent and noise-free motion representation, reducing unexpected robot movements and enabling smoother interaction.
Read more6/5/2024