Meta recently announced that the computing power required for training large-scale language models will increase exponentially in the future. Meta CEO Zuckerberg revealed during the earnings call that the training calculation volume of Llama 4 will be ten times that of Llama 3. This news highlights the high cost and technical challenges of large-scale language model training, and also reflects the continued fierce competition among technology giants in the field of AI. Meta is actively investing in building corresponding computing capabilities to meet future model training needs.
Meta, as the developer of the large-scale open source basic language model Llama, believes that the computing power required to train models will increase significantly in the future. On Tuesday, during Meta's second-quarter earnings call, Mark Zuckerberg revealed that training Llama4 will require 10 times the computing power of training Llama3. He emphasized that even so, Meta must build the ability to train models so as not to fall behind its competitors.
"Training Llama4 may require almost 10 times the amount of computation required to train Llama3, and future models will continue to require more computation," Zuckerberg said. He also noted that it is difficult to predict the development trend of future multi-generation models. , but at this moment it is better to build the required capabilities in advance than to be too late. After all, starting a new inference project requires a long preparation time.
In April this year, Meta released Llama3 with 80 billion parameters. Last week, the company released its upgraded version Llama3.1405B, with parameters reaching 405 billion, becoming Meta's largest open source model.
Meta's CFO Susan Lee also said that the company is considering different data center projects and building capabilities for training future AI models. Meta expects the investment to increase capital spending in 2025, she said.
You know, training large language models is a costly business. In the second quarter of 2024, Meta's capital expenditures increased by nearly 33% to $8.5 billion from $6.4 billion a year ago, driven by investments in servers, data centers and network infrastructure.
Highlights:
?Meta The computing power required to train Llama4 is approximately 10 times that of training Llama3.
?Meta expects investments in building capacity will increase capital expenditures in 2025.
?Training large language models is expensive, and Meta’s capital expenditures increased significantly in the second quarter.
All in all, Meta's huge investment in future AI model training demonstrates its ambitions in the field of artificial intelligence, but it also heralds the increasingly fierce competition in the AI industry and the rapid advancement of technology development. This will have profound consequences for the entire technology industry.