The improvement of large model reasoning efficiency is a key challenge in the field of artificial intelligence. High reasoning load, high cost and long response time seriously restrict the application of large models. In order to solve these problems, Kimi cooperated with Tsinghua University MadSys Laboratory to jointly develop the MoonCake reasoning system based on KVCACHE, and was officially released in June 2024. The system uses an innovative PD separation architecture and the concept of renewal calculation, which significantly enhances the inference throughput. To promote technology applications and popularization, the MoonCake project is officially open source.
Kimi Company and Madsys Laboratory of Tsinghua University launched the KVCACHE -based MoonCake reasoning system design solution, which was officially released in June 2024.
The MoonCake reasoning system has significantly enhanced the throughput of reasoning through the innovative PD separation architecture and the concept of renewal calculation, attracting extensive industry attention. In order to further promote the application and popularization of this technical framework, Kimi and Tsinghua University MadSys Laboratory jointly launched a multi -enterprise, such as 9#Aisoft, Alibaba Cloud, Huawei Storage, etc., and launched the open source project Mooncake. On November 28, MoonCake's technical framework was officially launched on the Github platform.
The MoonCake open source project revolves around the large -scale KVCACHE cache pool, and is committed to the MoonCake Store, which is dedicated to gradually open source and high -performance through stages. At the same time, the project will be compatible with multiple reasoning engines and underlying storage and transmission resources.
At present, the part of the transmission engine Transfer Engine is already open to the world on Github. The ultimate goal of the MoonCake project is to build a standard interface for new high -performance memory storage for the era of the big model, and provide relevant reference implementation solutions.
Xu Xinran, vice president of Kimi's engineering vice president, said: "By working closely with the Madsys Laboratory of Tsinghua University, we jointly created a separated large -model reasoning architecture MoonCake to achieve the ultimate optimization of reasoning resources.
MoonCake not only improves the user experience, but also reduces costs, providing effective solutions for handling long text and high and high -release needs. "He looks forward to more enterprises and research institutions to join the MoonCake project to explore more efficient model reasoning system architectures, so that AI assistants and other large model -based products can benefit wider people.
Project entrance: https: //github.com/kvcache- ai/moonCake
Points:
Kimi and Tsinghua University jointly released the MoonCake reasoning system to improve the efficiency of AI reasoning.
The MoonCake project has been opened on Github, which aims to build a high -performance memory storage standard interface.
Looking forward to the participation of more enterprises and research institutions to jointly promote the progress of AI technology.
The launch of the MoonCake open source project marks that the architecture of the large model reasoning system has moved towards a new stage. Its efficient performance and open cooperation model will effectively promote the progress and application of artificial intelligence technology, and contribute to the construction of a more intelligent world.