Introduction

Stability AI, a leading AI startup, has collaborated with chipmaker Arm to bring its Stable Audio Open model to mobile devices running on Arm chips. This innovation allows for the generation of audio, including sound effects, directly on mobile devices without the need for cloud processing.

Background

Most AI-powered apps that generate audio rely on cloud processing, which means they cannot be used offline. Furthermore, some audio generation models have been trained on copyrighted content, posing intellectual property risks. Stability AI's Stable Audio Open model, however, claims to have been trained entirely on royalty-free audio and songs, mitigating these risks.

Optimization and Demo

The optimized Stable Audio Open model, which will be demoed at the Mobile World Congress conference in Barcelona, can generate a sound from a text description, such as 'Gentle ocean waves at sunset.' Stability AI worked closely with Arm to optimize and 'distill' the model, resulting in a significant speedup of generation times by 30 times. Generating a single 11-second audio sample takes approximately 8 seconds on an Armv9 CPU.

Future Plans

Although the optimized Stable Audio Open model is not currently available for download, Stability AI's CEO, Prem Akkaraju, hinted at plans to bring the company's models, including Stable Audio Open, to consumer apps and devices in the future. This move is part of the company's strategy to make its models and workflows available to builders and creators everywhere.

Company Background

Stability AI, the company behind the popular image generation model Stable Diffusion, has faced challenges in the past, including financial difficulties and staff resignations. However, with new investments and a revamped leadership team, the company is working to turn its business around. Recent developments include the hiring of a new CEO, the appointment of James Cameron to its board of directors, and the release of several new image generation models.