A research team from the University of California, San Diego and MIT recently launched an open source remote operating system project called Open-TeleVision, a technology that can be regarded as a revolutionary breakthrough in the field of remote control. The system can realize precise robot control 3,000 miles away, making the operator feel as if he is in the scene in the science fiction movie "Avatar", and realizes remote control of the physical world through virtual reality technology.
The Open-TeleVision system demonstrates excellent device compatibility and supports a variety of terminal devices including Vision Pro, Quest series, Mac, iPad, and iPhone. Its core advantage lies in real-time three-dimensional video streaming technology, which provides users with accurate spatial perception and in-depth information, making the remote control experience as smooth and natural as flowing clouds. This seamless cross-platform connection feature greatly reduces the threshold for user use and paves the way for the popularization and application of remote control technology.
The core concept of this system is to achieve deep immersion of human-computer interaction through virtual reality technology. Users only need to wear VR headsets to achieve remote control of the robot without other complex wearable devices. The system cleverly simulates the human binocular stereoscopic vision and neck movement mechanism, allowing the operator to have an "immersive" control experience, and elevates human-computer interaction to a whole new level.
In terms of technological implementation, the Open-TeleVision system has shown many innovative highlights. First, it breaks through the limitations of traditional 2D video and presents 3D stereoscopic video through VR headsets, providing operators with accurate spatial distance perception. Secondly, the system is equipped with a moving neck simulation mechanism, and users can observe from multiple angles by simply turning their heads, greatly improving the convenience of control. In addition, the system adopts intelligent image processing technology and focuses on processing in the center of the picture, which significantly improves the computing efficiency and system frame rate.
In terms of specific implementation methods, the system adopts advanced inverse kinematics algorithms to accurately map the API provided by the VR headset to the robot to achieve accurate transmission of actions. The architecture design based on the web platform allows the system to be accessed on the browser of any device anytime, anywhere, achieving a low-latency and high-efficiency remote operation experience.
In response to the technical challenges in the field of remote operation, the Open-TeleVision system provides innovative solutions. Through the Mapping process, the system effectively solves the problem of mismatch between humanoid robots and human body degrees of freedom, and achieves widespread adaptation to various types of robots. At the same time, the system uses remote operation to collect training data, which significantly improves the robot's autonomous task completion ability and enables the system to show excellent reliability and generalization ability.
The project has been open sourced on the GitHub platform, with the project address at https://github.com/Improbable-AI/VisionProTeleop, providing valuable reference for the research and application of remote control technology. The launch of the Open-TeleVision system not only demonstrates the perfect integration of virtual reality technology and robotics technology, but also opens up new directions for the development of remote control, and is expected to play an important role in many fields such as industrial manufacturing, medical surgery, and dangerous environment operations. effect.