Meta Corporation recently released a new text generation 3D model technology called Meta3D Gen (3DGen), which can quickly and efficiently create high-quality 3D models with realistic textures and materials. This breakthrough technology combines the two core components of Meta3D AssetGen and Meta3D TextureGen to significantly improve the efficiency and quality of 3D model generation, bringing an unprecedented experience to users. 3DGen surpasses existing solutions in speed and quality, bringing new possibilities to areas such as game development, virtual reality, and the Metaverse.
Recently, researchers have released a new, fast text-generated 3D model technology called Meta3D Gen (3D Gen). 3D Gen creates 3D models in an efficient way with high-quality textures and materials, bringing users A whole new experience.
This new system combines two key components: Meta 3D AssetGen for creating 3D meshes and Meta 3D TextureGen for generating textures. These technologies work together to produce 3D assets with high-resolution textures and physically-based rendering (PBR) materials. According to Meta, the process is three to ten times faster than existing solutions.
Meta3D AssetGen
Among them, Meta3D AssetGen (AssetGen) is a major breakthrough in the field of text generation 3D, which can achieve high-quality 3D asset generation in less than 1 minute. AssetGen generates physically based rendering (PBR) materials to achieve powerful text-to-3D and text-to-texture generation capabilities with real lighting effects.
AssetGen first generates several views of the object, including appearance information for shadow and reflection channels, and then reconstructs color, metallicity, and roughness in 3D space, using a deferred shading loss for efficient supervision. It also uses signed distance functions to represent 3D shapes more reliably and introduces corresponding losses for direct shape supervision. To improve memory efficiency, this method uses fused convolution kernels. After extracting the mesh, a transformer that refines the texture in UV space significantly improves the clarity and detail of the texture.
AssetGen achieves a 17% improvement in Chamfer Distance and a 40% improvement in LPIPS compared to the best competing methods; at the same time, in human preference tests against the best industry competitors at the same speed, it achieved 72% favored.
Meta3D TextureGen
The researchers also introduced Meta3D TextureGen, a new forward approach consisting of two sequential networks designed to generate high-quality and globally consistent textures for geometry of arbitrary complexity in less than 20 seconds.
This method achieves state-of-the-art results in quality and speed by combining text-generated image models with 3D semantic conditions in 2D space and fusing them into a complete and high-resolution UV texture map.
The researchers also introduced a texture enhancement network that can upscale any texture at any scale to generate textures at 4K pixel resolution.
Official blog: https://ai.meta.com/research/publications/meta-3d-gen/
Highlights:
-Meta 3D Gen system creates high-quality 3D assets in less than a minute
- Meta3D Gen integrates the two core technologies of Meta3D AssetGen and Meta3DTextureGen.
- AssetGen supports generating physically based rendering materials with realistic relighting effects.
- Meta3D TextureGen is a method for quickly generating high-quality textures for geometry of any complexity.
The emergence of Meta3D Gen marks significant progress in text generation 3D model technology. Its efficiency, high quality and ease of use will greatly promote the development of the field of 3D content creation and bring unlimited possibilities to all walks of life. In the future, we can look forward to the application and further optimization of Meta3D Gen in more fields.