Meta FAIR Lab recently released a breakthrough multimodal fingertip tactile perception technology called "Sparsh", which is expected to completely revolutionize the field of robot manipulation. The Sparsh model uses self-supervised learning to pre-train more than 460,000 tactile images, and learn general tactile representations without manual labeling of data, realizing the robot's tactile perception ability that is similar to humans. It is compatible with a variety of visual haptic sensors and performs well in multiple tasks such as force estimation and sliding detection, especially when data is limited, its performance far exceeds that of traditional models.
The Sparsh model adopts self-supervised learning, using more than 460,000 tactile images for pre-training, and learns general tactile representations without manual labeling of data.
The model is able to support a variety of types of visual haptic sensors, including DIGIT, GelSight2017 and GelSight Mini, and significantly improves robot performance on tactile perception tasks such as force estimation, sliding detection, pose estimation, grab stability prediction and Fabric identification, etc. The researchers also constructed a standardized benchmarking platform called TacBench to evaluate the performance of different haptic sensors and models on various tasks.
Test results show that the Sparsh model performs well in all six tasks in TacBench, especially when the data volume is limited, and its performance is far beyond traditional task-specific and sensor-specific models. For example, in force estimation and sliding detection tasks, the Sparsh model can achieve satisfactory results even if only 1% of the annotation data is used. This means that Sparsh can help robots better understand the physical properties of objects and perform more refined manipulation.
The release of the Sparsh model marks a major breakthrough in the field of AI haptic perception. In the future, with the accumulation of more data and further optimization of models, Sparsh is expected to completely change the way robots interact with the physical world and promote the application of robotics in a wider range of fields.
Paper address:
https://scontent-sjc3-1.xx.fbcdn.net/v/t39.2365-6/464969941_1107633400780143_7479102347328147009_n.pdf?_nc_cat=103&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=y8Ui1HEw3BQQ7kNvgFe-ePu&_nc_zt=14&_nc_ht=scontent-sjc3-1. xx&_nc_gid=AeaFsuZziasVwPfMQsEoZqu&oh=00_AYAMqxGq0ATCySDxZWB0ZT8BgSkogYmj13c9f3ytVtkmSg&oe=672DEEE4
The emergence of Sparsh technology indicates that robotic haptic perception technology has reached a new milestone. Its potential in refined manipulation and human-computer interaction is huge, and its future application prospects are worth looking forward to.