An energy-efficient text-to-audio AI

Generative synthetic intelligence (AI) programs will encourage an explosion of creativity within the music trade and past, based on the University of Surrey researchers who’re inviting the general public to check out their new text-to-audio mannequin.

AudioLDM is a brand new AI-based system from Surrey that enables customers to submit a textual content immediate, which is then used to generate a corresponding audio clip. The system can course of prompts and ship clips utilizing much less computational power than present AI programs with out compromising sound quality or the customers’ capacity to govern clips.

The general public is ready to check out AudioLDM by visiting its Hugging Face area. Their code can also be open-sourced on GitHub with 1000+ stars.

Such a system might be utilized by sound designers in quite a lot of purposes, equivalent to film-making, game design, digital artwork, digital actuality, metaverse, and a digital assistant for the visually impaired.

Haohe Liu, venture lead from the University of Surrey, mentioned, “Generative AI has the potential to transform every sector, including music and sound creation.”

“With AudioLDM, we show that anyone can create high-quality and unique samples in seconds with very little computing power. While there are some legitimate concerns about the technology, there is no doubt that AI will open doors for many within these creative industries and inspire an explosion of new ideas.”

Surrey’s open-sourced mannequin is inbuilt a semi-supervised method with a way known as Contrastive Language-Audio Pretraining (CLAP). Utilizing the CLAP technique, AudioLDM might be educated on huge quantities of numerous audio information with out textual content labeling, considerably enhancing mannequin capability.

Wenwu Wang, professor in signal processing and machine studying on the University of Surrey, mentioned, “What makes AudioLDM special is not just that it can create sound clips from text prompts, but that it can create new sounds based on the same text without requiring retraining.”

“This saves time and resources since it doesn’t require additional training. As generative AI becomes part and parcel of our daily lives, it’s important that we start thinking about the energy required to power up the computers that run these technologies. AudioLDM is a step in the right direction.”

The user community has created a variety of music clips utilizing AudioLDM in several genres.

AudioLDM is a analysis demonstrator venture and depends on the present UK copyright exception exemption for information mining for non-commercial analysis. The paper is printed on the arXiv preprint server.