Adobe Research collaborated with Northwestern University to develop an artificial intelligence system called Sketch2Sound. This AI tool is expected to revolutionize the field of sound design. It allows users to create professional sound effects and ambient sounds just by humming, voice imitation, or simple text descriptions, greatly simplifying the sound design process and improving efficiency. Sketch2Sound analyzes the volume, timbre, and pitch of user input, combined with text descriptions, to intelligently generate the required sounds, such as identifying the bird songs imitated by the user and integrating them into the "forest atmosphere" sound effect.
Recently, Adobe Research collaborated with Northwestern University to develop an artificial intelligence system called Sketch2Sound. This tool is expected to completely change the way sound designers work. Sketch2Sound enables users to create professional sound effects and ambiences by humming, imitating sounds, and using simple text descriptions.
The system analyzes three key elements of the user's vocal input: volume, timbre (which determines how bright or dark the sound is) and pitch. It then combines these features with the user's textual description to generate the desired sound. For example, when a user enters "forest atmosphere" and makes short sounds, the system automatically recognizes these sounds as birdsong without specific instructions.
Another great thing about Sketch2Sound is its ability to understand context. When making music, users can enter "bass drum, snare drum" and hum the rhythm. The system intelligently places the bass drum on low notes and the snare drum on high notes. This intelligent processing greatly simplifies the sound design process.
In order to meet the needs of professionals, the research team also built in special filtering technology, allowing users to adjust the accuracy of the generated sound according to their needs. Sound designers can choose between very precise control or a more relaxed, approximate approach, and this flexibility may make Sketch2Sound especially popular with Foley artists. Using this tool, professionals who create sound effects for movies and TV shows can more quickly create effects using sound and text descriptions, rather than having to manipulate physical objects to create sounds.
While the researchers note that spatial audio characteristics in input recordings can sometimes have an adverse effect on the resulting sound, they are working to address this issue. Currently, Adobe has not announced whether Sketch2Sound will be launched as a commercial product or when it will be released.
Project entrance: https://hugofloresgarcia.art/sketch2sound/
Highlights:
Sketch2Sound is a newly developed AI tool that creates sound effects through humming and text description.
The system analyzes volume, timbre and pitch, combining the user's vocal input with text to generate targeted sound effects.
Especially suitable for Foley artists, it can quickly generate film and television sound effects and improve work efficiency.
All in all, with its intelligence and convenience, Sketch2Sound has the potential to become a powerful assistant for sound designers and Foley artists, greatly improving work efficiency. Although it is still in the research and development stage, its future development is worth looking forward to. The project link has been provided for interested users to learn more.