Meta recently announced the launch of the latest member of its Llama series - Llama3.370B. Ahmad Al-Dahle, Meta’s vice president of generative AI, broke the news on the Significantly reduced.
Al-Dahle said that by leveraging the latest post-training technology, Llama3.370B not only improves performance but also reduces operating costs. According to benchmark test results released by Meta, Llama3.370B has surpassed Google's Gemini1.5Pro, OpenAI's GPT-4 and Amazon's newly released Nova Pro in many fields, especially in the MMLU test that evaluates the model's language understanding ability.
The model is now available for download from sources such as Hugging Face and Llama’s official platform, a move by Meta that aims to dominate the AI field with “open” models. Meta's Llama model can be applied to a variety of scenarios and supports commercialization, although Meta has set usage restrictions for some developers, requiring platforms with more than 700 million monthly users to apply for special licenses. Despite this, the Llama model has been downloaded more than 650 million times, indicating its widespread popularity among AI developers around the world.
In order to support larger-scale AI model training in the future, Meta is investing heavily in computing infrastructure. The company recently announced that it will build a $10 billion AI data center in Louisiana, which is Meta’s largest AI data center to date. Zuckerberg mentioned during the earnings call that in order to train the next-generation Llama4 model, the computing power required will be 10 times that of Llama3. Meta has currently purchased more than 100,000 Nvidia GPU clusters, matching the resources of competitors such as xAI.
As the cost of generative AI model training continues to rise, Meta's capital expenditures are also showing an increasing trend. Capital expenditures in the second quarter of 2024 increased by nearly 33% to US$8.5 billion. This growth is mainly due to Meta's continued investment in servers, data centers and network infrastructure.