Tencent today announced the open source of its large language model called Hunyuan-large, with a parameter scale of 398B and a 52B activation parameter volume. The model performs well in multiple authoritative benchmarks, surpassing similar open source models like Llama 3.1 and Mixtral. Its technological innovations include the application of high-quality synthetic data, which effectively solves the problem of insufficient natural data and supports text sequences up to 256K, significantly improving long text processing capabilities. In addition, Tencent has also opened the evaluation data set called "Penguin Scroll", aiming to make up for the lack of high-quality long text evaluation sets in the industry and promote the development of big model technology.
Tencent today released the open source MOE large language model Hunyuan-large, with a total parameter volume of 398B and an activation parameter volume of 52B. Public evaluation results show that Tencent Hunyuan Large is leading in CMMLU, MMLU, CEva1, MATH and other multidisciplinary comprehensive evaluation sets, as well as nine dimensions such as Chinese and English NLP tasks, code and mathematics, surpassing first-class open source majors such as Llama3.1 and Mixtral. Model.
It is understood that this model can achieve high-quality synthetic data in technological innovation, and effectively deal with the shortcomings of natural data through the use of synthetic data. In terms of context processing capabilities, the pre-trained model supports text sequences up to 256K, significantly enhancing the ability to handle long context tasks.
At the same time, Tencent Hunyuan announced that in order to fill the shortcomings of real long article review sets in the industry, Tencent Hunyuan will soon open source Penguin Scroll Review Set to help industry applied research. The self-developed Penguin Scrolls are based on a variety of natural long texts such as public finance, law, and academic papers. The length range is 1K-128K, covering various deep reading comprehension and long-text inference tasks.
The release of Tencent Hunyuan Large large language model and the open source of the Penguin Scroll evaluation set will provide the industry with more powerful language models and evaluation tools to promote the development of natural language processing and artificial intelligence.
Official website address: https://llm.hunyuan.tencent.com
The open source of Hunyuan-large and the simultaneous release of the Penguin Scroll Review Set mark another major breakthrough for Tencent in the field of large language models, providing strong support for academic research and industrial applications, and it is worth looking forward to its future development of artificial intelligence. role in.