The Colossal-AI team has open sourced the Sora algorithm reproduction solution Open-Sora, aiming to reduce reproduction costs and improve performance. This solution covers the entire process from data processing to training inference, supports multiple model structures, video compression methods and parallel training optimization, significantly improving training efficiency. Open-Sora used the DiT-XL/2 model to conduct a performance test of 600K sequence length on a single H800 SXM 8*80GB GPU. The results showed that its performance improvement and cost reduction exceeded 40%. This project is committed to providing a complete and efficient Sora reproduction solution to provide convenience for researchers and developers.
Colossal-AI has open sourced the complete Sora recurrence architecture solution Open-Sora, claiming to reduce the recurrence cost by 46% and expand the model training input sequence length to 819K patches. Sora algorithm reproduction solution In Sora's technical report, Sora uses a video compression network to compress videos of various sizes into a sequence of spatio-temporal blocks in a latent space, then uses a Diffusion Transformer for denoising, and finally decodes to generate a video. Open-Sora summarizes the training pipeline that Sora may use to provide a complete Sora recurrence architecture solution, including the entire process from data processing to training inference. At present, Open-Sora has provided a complete Sora recurrence architecture solution, including the entire process from data processing to training inference, supporting dynamic resolution, multiple model structures, multiple video compression methods, and multiple parallel training optimizations. In terms of performance, taking the performance test of the DiT-XL/2 model on a single H800 SXM 8*80GB GPU as an example, at a sequence length of 600K, the Open-Sora solution has more than 40% performance improvement and cost compared to the baseline solution. reduce. Open-Sora open source address: https://github.com/hpcaitech/Open-Sora.Open-Sora's open source provides valuable resources to researchers and developers, accelerating the advancement of video generation technology. Its high efficiency and low cost are expected to encourage more people to participate in research in this field. We look forward to further improvement and development of Open-Sora in the future to contribute more to the field of AI video generation.