In the field of image retrieval, how to quickly and accurately find target images has always been a hot research topic. Traditional text-based retrieval methods are difficult to cope with diverse query styles such as sketches and artistic paintings. Peking University's Yuan Li research group, together with researchers from Nanyang Technological University and Tsinghua Institute of Automation, proposed a breakthrough method of "universal style retrieval" and built the corresponding FreestyleRet framework to achieve accurate retrieval of images of different styles. It has brought revolutionary progress to image retrieval technology, greatly improved retrieval efficiency and accuracy, and provided users with a more convenient and smarter image search experience.
In the digital age, we deal with massive images every day. But have you ever thought about how amazing it would be to be able to quickly find the picture we want through a sketch, an art painting, or even a blurry photo? Yuan from Peking University The Li research group, together with researchers from Nanyang Technological University and Tsinghua Institute of Automation, have brought us such a surprise - a brand-new image retrieval technology that can cope with diverse query styles, whether it is sketches or artistic paintings. Even low-resolution images can be accurately matched.
The core of this technology is the "universal style retrieval" method they proposed. It is different from traditional text-based image retrieval. The new method can handle multiple query styles and even combined queries, such as sketches plus text, art paintings plus text, etc. This not only improves the flexibility of retrieval, but also greatly improves the accuracy of retrieval.
To achieve this goal, the research team constructed two unique datasets: DSR (Diverse-Style Retrieval Dataset) and ImageNet-X. DSR contains 10,000 natural images and corresponding texts of four retrieval styles, while ImageNet-X contains 1 million natural images with various style annotations. The establishment of these two data sets provides rich training and testing resources for new methods.
What’s even more exciting is that the research team also proposed a framework called FreestyleRet. This framework effectively solves the problem of existing models being incompatible with different types of retrieval vectors by extracting image style and injecting it into the retrieval model. The FreestyleRet framework consists of three main modules: a style extraction module, a style space building module, and a style-inspired hint fine-tuning module. These modules work together to enable the retrieval model to understand and process various styles of query vectors.
In experiments, the FreestyleRet framework demonstrated excellent performance. It not only achieves significant improvements in Recall@1 and Recall@5 performance on DSR and ImageNet-X datasets, but also shows good generalization capabilities and scalability when processing multiple different styles of query vectors. .
The results of this research have been published publicly, and the detailed paper can be viewed on arXiv. At the same time, the relevant code and data sets have also been open sourced for further exploration and application by interested researchers and developers.
This is not only a technological leap in the field of image retrieval, but also a huge convenience for each of us in our daily lives. Imagine that in the future, whether we are looking for inspiration, conducting academic research, or daily entertainment, we will be able to find the image resources we need more quickly and accurately. This is the power of technology that makes everything possible.
Paper address: https://arxiv.org/pdf/2312.02428
This image retrieval technology based on "universal style retrieval" brings new possibilities to future image searches. Its open source code and data sets will promote further development in this field, provide technical support for more application scenarios, and make it more convenient. The era of efficient image search.