In recent years, there has been a growing demand for automated image processing technology in the field of digital art. The research team of Tsinghua University and Tencent ARC Laboratory proposed a new model called ColorFlow to solve the problem of coloring black and white image sequences. This model is designed to solve the difficult problem of colorizing black and white image sequences while maintaining character and object identity consistency, which has important practical implications for industries such as comics and animation. The ColorFlow model not only improves coloring efficiency and quality, but also simplifies the complex color generation process, bringing new possibilities to digital art creation.
With the development of digital art, automated image processing technology has attracted increasing attention. Recently, a research team from Tsinghua University and Tencent ARC Laboratory proposed a new image sequence coloring model called ColorFlow. This model aims to solve the problem of maintaining the identity consistency of characters and objects while colorizing black and white image sequences, meeting the practical needs of industries such as comics and animation.
ColorFlow is a three-stage diffusion-based framework that leverages contextual information to accurately generate colors for black and white image sequences from a pool of reference images. For example, the model can effectively color a character’s hair color and clothing, ensuring color consistency with reference images. Unlike previous technologies that required fine-tuning for each role, ColorFlow simplifies the color generation process through an innovative search-enhanced color pipeline with strong generalization capabilities.
The model was designed with two main branches: one for extracting color identity, and another responsible for the actual coloring process. This dual-branch design takes full advantage of the diffusion model and is able to achieve powerful context learning and color identity matching through the self-attention mechanism. In order to verify the effectiveness of ColorFlow, the research team also launched ColorFlow-Bench, a comprehensive benchmark test specifically for reference image-based coloring tasks.
In comparative experiments, ColorFlow surpassed existing advanced models in multiple indicators, demonstrated higher aesthetic quality, and generated colors that were closer to the original image. The research team demonstrated the application effects of ColorFlow in different art scenarios, including black and white comics, line art, real-world photos and cartoon storyboards, and achieved satisfactory results.
The launch of ColorFlow not only sets a new benchmark for automatic coloring technology of image sequences, but also provides strong support for the further development of the art industry. The research team hopes that this technology can be more widely promoted in practical applications and promote innovation and progress in digital art creation.
Project entrance: https://zhuang2002.github.io/ColorFlow/
Highlights:
ColorFlow is an innovative black-and-white image sequence coloring model that maintains character identity consistency.
This model adopts a dual-branch design, which is used for color identity extraction and actual coloring respectively, improving the effect and efficiency of coloring.
ColorFlow surpasses existing state-of-the-art models in multiple metrics, demonstrating higher aesthetic quality and practicality.
The emergence of the ColorFlow model marks significant progress in automatic coloring technology for image sequences. Its efficient and accurate coloring capabilities and strong generalization capabilities provide strong technical support for animation, comics and other industries. It is expected to be applied in more fields in the future and promote the vigorous development of digital art creation.