ByteDance and researchers from the University of Shanghai Science and Technology jointly developed an innovative technology called "HeadGAP" that can quickly generate a realistic and animatable 3D virtual human head model using only three photos of a target person from different angles. . This technology breaks through traditional 3D modeling's reliance on large amounts of data and complex operations, brings unprecedented convenience to the creation of virtual characters, and has extremely high personalized customization capabilities. Through the 3D head prior information accumulated in the "prior learning" stage, HeadGAP can efficiently generate virtual avatars that highly restore the characteristics of the target character, and support facial expression synchronization based on reference videos.
Recently, researchers from ByteDance and ShanghaiTech University developed a study called "HeadGAP" that attracted everyone's attention. The research team proposed a new method that can only use three pictures of different perspectives of the target person. Using photos, you can quickly create a highly realistic, animatable 3D virtual human head image, and synchronize facial expressions based on reference videos.
The research team demonstrated how to create personalized avatars with a small amount of data in real scenarios. In this study, the researchers first conducted a stage called "prior learning." At this stage, they extracted prior information about the 3D head from a large multi-view dynamic dataset. This prior information can help the system understand different head features and expressions. Then, in the "avatar creation" stage, researchers use this prior information to perform personalized customization and generate a virtual avatar of the target person.
The entire process uses a self-decoding network based on Gaussian point clouds, combined with partial dynamic modeling. In this way, the system can quickly capture the uniqueness of each individual and perform personalized optimization of the avatar on this basis. The team also used some technical means, such as inversion and fine-tuning strategies, to make the avatar personalization process more efficient, and ultimately successfully achieved photo-level rendering effects and multi-view consistency.
In experiments, the research team demonstrated the performance of their method in different scenarios. The results showed that the generated 3D avatars can maintain high quality and stable animation effects whether in a controlled environment or in a real environment. This achievement not only has broad application prospects in virtual social networking, game development and other fields, but also provides new ideas and methods for the personalized production of 3D avatars.
Product entrance: https://top.aibase.com/tool/headgap
Highlight:
The research team used the "HeadGAP" method to create a realistic 3D head virtual image with only a few photos.
This method uses Gaussian point network and dynamic modeling technology to achieve personalized customization and optimization of avatars.
Experimental results show that the generated avatar has excellent rendering quality and animation performance, and is suitable for a variety of application scenarios.
The emergence of HeadGAP technology marks a new milestone in 3D virtual portrait generation technology. Its efficiency, convenience and high fidelity will surely bring revolutionary impact to fields such as virtual reality and the metaverse. In the future, we can expect this technology to be used in more fields to bring people more realistic and personalized virtual experiences.