On September 5, Shun Xiangyang, chairman of the Board of Trustees of the Hong Kong University of Science and Technology and foreign academician of the National Academy of Engineering, shared his eight thoughts on the implementation of the large-scale model industry at the 2024 Inclusion·Bund Conference. He believes that the arrival of the AI Agent era will not be a magical and powerful model that suddenly replaces all workflows. It involves the continuous integration of technology, engineering and market, and finally presents services to humans that exceed expectations.
Thought 1: Computing power is the threshold "Today, when doing large models and deep learning, the first and most important thing is to have computing power." Shen Xiangyang said. He pointed out that since 2010, the computing power required for large models has increased by 6 or 7 times. It has stabilized in the past few years and has grown approximately 4 times every year. The model is getting larger and larger, the number of parameters is getting larger and larger, and the demand for computing power is also growing in a flat direction as the parameters increase. In his view, the development of the entire computer chip industry has changed from the original "Moore's Law" to "Huang's Law." Moore's Law used to hold that computing power doubled every 18 months. It is now predicted that GPU will drive AI computing power to double year by year. "It hurts feelings to talk about cards, but there is no emotion if you don't have cards. There used to be a saying that poverty limits imagination, but now poverty may distort imagination, because if there are no cards, the projects that can be imagined may be different." Shen Xiangyang sighed with emotion. road. Thought 2: Data about data Public information shows that the training data of GPT3 has reached a token (throughput) of 2 T, and that of GPT4 has reached about 12 T. According to Shun Xiangyang's prediction, the training data of GPT5 may reach 200 T. The current data on the Internet is far from meeting the needs of future model training, and we need to think about ways to mine more data. In the field of artificial intelligence, data is regarded as the "fuel" of the model, and the model needs to learn and extract useful information from this data. Therefore, the quantity, quality and diversity of data will directly affect the accuracy and performance of the model. Shen Xiangyang said that in the past, as the core accumulation of the Internet, most of the data was used by Google to make search engines. In the future, these data will be used to train large models. "The data accumulated by the Internet over the past 40 years seems to be just for such an AI moment." Thought 3: The next chapter of the big model What’s next? Shen Xiangyang believes that the future development path of the large model industry is very clear, and it will move from the previous large language model to the multi-modal model and towards the world model in the future. Technically speaking, we must take the path of unifying understanding and generation. "The future will definitely move in the direction of embodied intelligence and robots. One of the special forms is autonomous driving." Shen Xiangyang said. In fact, there is no standard definition of world model in the industry. The sora model launched by OpenAI has triggered discussions on the "world model" in the industry. OpenAI regards it as the basis for models that can understand and simulate the real world, and believes that its capabilities are an important milestone in achieving AGI (artificial general intelligence). However, Shun Xiangyang believes that "Although the Sora model is very good, it is not that powerful. The physical properties in it cannot be guaranteed, and it cannot be a world model." Thought 4: Large models sweep across thousands of industries Large models can be divided into general large models, industry large models, enterprise large models and personal large models. Shen Xiangyang pointed out that general-purpose large models are the basis of AI, and training a general-purpose large model requires at least 10,000 calories; industry large models are the base for domain applications and require kilocalorie-level training; enterprise large models are the rediscovery of the value of enterprise data , requires hundreds of calories of training. These large models have extremely high requirements on computing power. "The most exciting thing is the large-scale personal model. For example, Lenovo and Microsoft are promoting AIPC and Apple's Apple Intelligence are all developing in the direction of personal intelligence." Shen Xiangyang said. As of the end of July this year, China has registered 197 large models, of which 30% are general large models and 70% are industry large models. "It can be seen that large models in the industry account for the vast majority, and there will definitely be more and more in the future." Shen Xiangyang said. Thought 5: ai agent—from vision to implementation In May 2024, Microsoft founder Bill Gates publicly stated that AI Agent will not only change the way everyone interacts with computers, but will also subvert the software industry and bring about the biggest computing revolution since typing commands to clicking icons. Shun Xiangyang agreed with this view. He believes that in the era of artificial intelligence, the truly amazing super application is AI Agent. In the process of AI Agent from vision to implementation, it is necessary to always focus on needs, deeply understand the capabilities of the model, and build a workflow with deep AI participation. "If you work in a company today, the entire workflow is very complex. Although ChatGPT is very powerful, it is far from reaching the level of Agent. It only achieves a single breakthrough. To truly move forward, it must be integrated into the entire workflow. ." he said. Thought 6: Pay attention to the governance of AI AI governance is very important. The theme of this year's World Artificial Intelligence Conference (WAIC) is about AI governance. Various countries have many different views on this matter. The development of AI has had a strong impact on people, companies, government supervision, social development and other aspects, and has triggered public concerns about its security governance. "I think the next important point in the development of artificial intelligence. From the perspective of various countries around the world, it is necessary to build sovereign artificial intelligence, and behind sovereign artificial intelligence there must be a sovereign cloud to support the development of sovereign artificial intelligence." Shen Xiangyang expressed. Thought 7: Rethink the human-machine relationship "How much of the impact brought by GPT is the shock of human-computer interaction, and how much is the development of machine intelligence?" Shen Xiangyang believes that the relationship between humans and machines should be rethought. He pointed out that AI provides humans with a new context for symbiosis with technology, and the new way of human-computer interaction points to the integration and progress of "AI and IA". IA (Intelligent Augmentation) represents a human-centered AI development path. It focuses on using technology to enhance human capabilities rather than replace humans, emphasizing the collaborative relationship between humans and AI. "New York Times columnist John Markoff mentioned that in the development of computers over the past few decades, the real winner has been in human-computer interaction. No matter what the technology is, the ultimate goal should be to help humans use machines better." Shen Xiang Yang said, "In the AI era, the most essential aspect of human-computer interaction is dialogue, just like ChatGPT. Will ChatGPT plus Microsoft become the greatest company in the AI era? I think only time will tell." Thought 8: The nature of intelligence Today, the development of GPT is in full swing, but in fact, people's understanding of intelligence is still very limited. Unlike physics, everything from the vast starry sky to tiny quanta can be explained by a unified theory; many things in today's deep learning are unexplainable and have no robustness. "The essence of intelligence is the century-old battle between neural networks and symbol systems." Shen Xiangyang said, "Today, although the development of artificial intelligence is still in a relatively early stage, there are already many applications in the industry, which are worthy of I am determined to do it and I am confident about the future. ”
