People can now transform every sound like whistling into something like a violin, or a saxophone, or a flute through a new tool introduced by Google. Called Tone Transfer, it is a new machine learning tool that helps users create music with everyday sounds.
"We created a tool called Tone Transfer to allow musicians and amateurs alike to tap into DDSP (Differentiable Digital Signal Processing) as a delightful creative tool," Nida Zada, UX Lead, Google Research, wrote in a blog post on Thursday.
An open source technology, DDSP is a new approach to machine learning that enables models to learn the characteristics of a musical instrument and map them to a different sound.
Developed by Google Research's Magenta team, the process can lead to many creative, quirky results. For example, one can try replacing a capella singing with a saxophone solo, or a dog barking with a trumpet performance. "This development is important because it enables music technologies to become more inclusive," Zada said.
Machine learning models inherit biases from the datasets they are trained on, and music models are no different. Many are trained on the structure of western musical scores, which excludes much of the music from the rest of the world.
Rather than following the formal rules of western music, like the 12 notes on a piano, DDSP transforms sound by modelling frequencies in the audio itself. This opens up machine learning technologies to a wider range of musical cultures, Zada said.
*Edited from an IANS report