For a long time, the agile movement of humanoid robots has been a huge challenge in the field of robot research. The physical differences between the simulated environment and the real world make it difficult for robots to directly apply the results of simulation training to reality. This article introduces a new framework called ASAP (Aligning Simulation and Real Physics), which effectively solves this problem by cleverly aligning simulation and real physical characteristics, allowing humanoid robots to achieve more flexible and coordinated full-body movement. .
For a long time, people have dreamed that humanoid robots can be as flexible as humans, and even surpass humans. However, due to the physical differences between the simulated environment and the real world, achieving full-body coordination and agile movement of the robot remains a huge challenge. Traditional methods of system identification and domain randomization often rely on cumbersome parameter adjustments, or lead to robot movements being too conservative, sacrificing agility. Now, a new framework called ASAP (Aligning Simulation and Real Physics) emerges. By cleverly aligning simulation and real physics, the shape robot can master more flexible full-body motor skills.
The ASAP framework is divided into two key stages. First, during the pre-training phase, researchers will use human motion video data to remap these actions onto the humanoid robot, and then train the robot to learn these actions in a simulated environment. However, applying the trained strategies in the simulated environment to real robots directly will often lead to performance degradation because there are dynamic differences between the simulated environment and the real world. To solve this problem, the ASAP framework enters the second phase - the post-training phase. At this stage, researchers will ask the robot to perform pre-trained actions in the real world and record the robot's actual motion trajectory.
Next, the ASAP framework will use this real-world motion data to reproduce the robot's motion in the simulator. Due to differences in the simulated environment and the real world, the simulated motion trajectory often deviates from the real motion trajectory. This difference just happens to provide researchers with a signal to learn. ASAP trains a "difference action model" that learns and compensates for dynamic differences between simulation and reality. This model is like an "error correction" that can correct shortcomings in the simulator and bring them closer to real-world physical characteristics. Finally, the researchers will integrate this "difference action model" into the simulator and use it to fine-tune the pre-trained motion tracking strategy, so that the robot's movements can better adapt to real-world physical properties. The fine-tuned strategy can be directly deployed to real-world robots without the need to use the "difference action model".
To verify the effectiveness of the ASAP framework, the researchers conducted several experiments, including migration between different simulators and testing on the real humanoid robot Unitree G1. Experimental results show that the ASAP framework significantly improves the agility and whole-body coordination of robots in various dynamic movements. Compared with traditional system identification, domain randomization and dynamic difference learning methods, ASAP can significantly reduce motion tracking. error.
The success of the ASAP framework is that it can effectively bridge the dynamic differences between the simulated environment and the real world, so that humanoid robots trained in the simulated environment can truly demonstrate superb agility in the real world, which makes development more flexible and The multi-functional humanoid robot points out a new direction.
Key technologies of the ASAP framework include:
Pre-training using human motion data: convert human agile movements into robot learning goals to provide robots with high-quality motion data.
Training of differential action models: By learning the differences between the real world and the simulation environment, dynamically compensate for the shortcomings of the simulator and improve the accuracy of the simulation.
Strategy fine-tuning based on differential action models: enables robot strategies to adapt to real-world physical characteristics and ultimately achieve higher motion performance.
Experimental verification of the ASAP framework shows that:
In migration between simulators, ASAP is able to significantly reduce motion tracking errors, which is superior to other benchmark methods.
In testing on real robots, ASAP can also significantly improve the robot's motion performance, allowing the robot to complete difficult agile movements.
The study also explores in-depth the key factors of training differential action models, including data set size, training duration and action norm weight. In addition, the researchers compared different differential action model usage strategies and finally confirmed that the reinforcement learning fine-tuning method can achieve optimal performance.
Despite the remarkable progress of the ASAP framework, it still has some limitations, such as hardware limitations, dependence on motion capture systems, and high data demands. Future research directions may include developing strategic architectures that can perceive hardware damage, leveraging mark-free pose estimation or onboard sensor fusion to reduce dependence on motion capture systems, and exploring more efficient adaptive techniques for differential action models.
The emergence of the ASAP framework has brought new hope to the field of humanoid robots. By cleverly solving the dynamic differences between simulation and reality, ASAP enables humanoid robots to master more agile and coordinated motor skills, laying a solid foundation for the widespread application of humanoid robots in the real world in the future.
Project address: https://agile.human2humanoid.com/
Paper address: https://arxiv.org/pdf/2502.01143
The ASAP framework provides an effective solution to solve the gap between humanoid robot simulation and reality. Its remarkable achievements in improving robot agility and coordination have pointed out the direction for the future development of humanoid robot technology and also indicates a more flexible and intelligent humanoid robot. It's coming soon. Future research can further optimize the ASAP framework to make it more robust and efficient in practical applications.