The editor of Downcodes learned that Tencent today released the open source MOE large language model Hunyuan-large, with a parameter size of 398B and an activation parameter size of 52B. This model has performed well in multiple authoritative benchmark tests, surpassing Llama3.1, Mixtral and other first-class open source in nine major dimensions, including CMMLU, MMLU, CEva1, MATH and other multi-disciplinary comprehensive evaluation sets, as well as Chinese and English NLP tasks, code and mathematics. Large model, showing powerful performance and wide application potential. The technological innovation of Hunyuan-large lies in the application of high-quality synthetic data, which effectively solves the problem of insufficient natural data and supports the processing of text sequences up to 256K, greatly enhancing the processing capabilities of long context tasks.
It is understood that this model can achieve high-quality synthetic data in terms of technological innovation. By using synthetic data to enhance training, it can effectively cope with the shortcomings of natural data. In terms of context processing capabilities, the pre-trained model supports text sequences up to 256K, significantly enhancing the ability to handle long context tasks.
At the same time, Tencent Hunyuan announced that in order to fill the shortage of real long-text review sets in the industry, Tencent Hunyuan will open source the Penguin Scroll review set to help industry application research. Self-developed PenguinScrolls is based on a variety of natural long texts such as public finance, law, and academic papers, with a length range of 1K-128K, covering various in-depth reading comprehension and long-text reasoning tasks.
The release of Tencent Hunyuan Large language model and the open source of the Penguin Scroll evaluation set will provide the industry with more powerful language models and evaluation tools, and promote the development of natural language processing and artificial intelligence.
Official website address: https://llm.hunyuan.tencent.com
The open source of Tencent's Hunyuan large model not only provides developers with powerful tools, but also contributes to the progress of the field of artificial intelligence. The open source of the Penguin Scroll review set will further promote the improvement and development of long text processing technology. Looking forward to more innovative results in the future!