The RWKV Open Source Foundation has released the RWKV-6-World14B model, which is currently one of the most powerful dense pure RNN large language models in the world. The model excels in multi-language capabilities, supporting over 100 languages and codes, and outperforms models such as Llama2 13B and Qwen 1.5 14B in multiple benchmarks. Its excellent performance stems from the improvement of the RWKV architecture, and avoids optimization for specific benchmark tests during the training process, ensuring the true capabilities and generalization capabilities of the model. Users can easily download and deploy the model through platforms such as Hugging Face, ModelScope and WiseModel.
On July 19, 2024, the RWKV Open Source Foundation announced the global open source of the RWKV-6-World14B model, which is currently the strongest dense pure RNN large language model. The model performed well in the latest performance test, with English performance equivalent to Llama213B, and significantly ahead in multi-language performance, supporting more than 100 languages and codes around the world.
The benchmark test of the model includes 4 open source large language models with a scale of nearly 14B parameters, 12 independent benchmark tests to evaluate English performance, and four benchmark tests of xLAMBDA, xStoryCloze, xWinograd and xCopa to evaluate multi-language capabilities. RWKV-6-World14B performed well in these tests, especially in the Uncheatable Eval ranking list, where the comprehensive evaluation score exceeded llama213B and Qwen1.514B.
The performance improvement of the RWKV-6-World14B model benefits from the architectural improvements from RWKV-4 to RWKV-6. This model did not add any benchmark test data sets during training, avoiding special optimization, so its actual ability is stronger than the scoring ranking. In the Uncheatable Eval evaluation, RWKV-6-World14B was evaluated on real-time data such as the latest arXiv papers, news, ao3 novels and GitHub codes released in July, showing its real modeling capabilities and generalization capabilities.
Currently, the RWKV-6-World14B model can be downloaded and deployed locally through platforms such as Hugging Face, ModelScope, and WiseModel. Since Ai00 only supports models in safetensor (.st) format, you can also download models that have been converted to .st format in the Ai00HF warehouse. The graphics memory requirements for locally deploying and inferring the RWKV-6-World14B model vary from about 10G to 28G depending on the quantification method.
The effect preview of the RWKV-6-World14B model includes natural language processing (sentiment analysis, machine reading comprehension), prose poetry and literary creation, reading and modifying codes, financial paper topic selection suggestions, extracting key content of news, one-sentence text expansion, and Write multiple application scenarios such as Python Snake game.
It should be noted that all open source released RWKV models are base models, which have certain command and dialogue capabilities, but have not been optimized for specific tasks. If you want the RWKV model to perform well on a specific task, it is recommended to use data sets of related tasks for fine-tuning training.
Project address:
Hugging Face: https://huggingface.co/BlinkDL/rwkv-6-world/tree/main
ModelScope:https://modelscope.cn/models/RWKV/rwkv-6-world/files
WiseModel: https://wisemodel.cn/models/rwkv4fun/Rwkv-6-world/file
In short, the open source of the RWKV-6-World14B model has brought new breakthroughs to the field of large language models. Its powerful performance and wide application prospects are worth looking forward to. Developers can download and conduct further exploration and application on various platforms according to their own needs.