Intelligent sound engineering for all

Signal processing challenges to rework music for those with a hearing loss

Intelligent sound engineering opens up the possibility of personalizing audio, for example processing and mixing music so the audio quality is better for someone with a hearing loss. People with a hearing impairment can experience problems when listening to music with or without hearing aids. 430 million people Worldwide have a disabling hearing loss, with this number increasing as the population ages. Poor hearing makes music harder to appreciate, for example picking out the lyrics or melody is more difficult. This reduces the enjoyment of music, and can lead to disengagement from listening and music-making.

I work on the Cadenza project, which has just launched a series of open competitions to get experts in music signal processing and machine learning to develop algorithms to improve music for those with a hearing loss. Such open challenges are increasingly used to push forward audio processing. They’re free to enter, and we provide lots of data, software and support to help competitors take part.

The Cadenza Challenges are about improving the perceived audio quality of recorded music for people with a hearing loss.

What do we mean by audio quality? Imagine listening to the same music track in two different ways. First on a low quality mp3 played on a cheap mobile, and then via a high quality wav and studio-grade monitors. The underlying music is the same in both cases, but the audio quality is very different.

Headphones

The first task you might tackle is our Task 1: listening over headphones. The figure below shows the software baseline that we are providing for you to build on. First the stereo music is demixed into VDBO (Vocals, Drums, Bass, Other) before being remixed into stereo for the listener to hear. At the remixing stage there is an opportunity for intelligent sound engineering to process the VDBO tracks and adjust the balance between them, to personalise and improve the music. We’re also hoping for improved demixing algorithms that allow for the hearing abilities of the listeners.

Car

The second task you could tackle is intelligent sound engineering in the presence of noise. Listening to music in the car against the rumble of car noise is really common. How would you tune a car stereo (Enhancement box in the diagram below), so the processed music is best allowing for both the noise and the simple hearing aid the driver is wearing?

Next steps

Both tasks are live now, with entrants having to finish and submit their entries in July 2023. Join us in trying to improve music for those with a hearing loss. Or let us know what you think below, e.g., what do you think of the project idea and the scenarios we’ve chosen.

You’ll find lots more on the Cadenza project website, including a Learning Resources section that gives you background information on hearing, hearing loss, hearing aids and other knowledge you might need to enter the challenge. We also have a “find a team” page, if you want to get together with other experts to improve music for those with a hearing loss.

Engine sounds for game engines

Engines and motors feature heavily in video games, but they can be very problematic. As game car audio expert Greg Hill said, “For engine sounds most people think we just record a car and throw the sound into a folder and it magically just works in the game. Game sounds are interactive – unlike passive sound like movies where the sound is baked onto an audio track… sound has to be recorded a certain way so it can be decomposed and reconstructed into a ‘sonic model’ and linked to the physics so the player has full control over every parameter.”

And therein lies the issue; recorded sounds are fixed, but the game sounds need to be adaptable and controllable. One can get around this, but only with huge effort.

So Nemisindo (the Zulu word for Sound Effects, our start-up company) have brought their deep and advanced engine sound generator to Unity as an audio plugin.

The Nemisindo Engine implements a flexible, controllable and realistic engine for all your game vehicles. It is a Unity native audio plugin and can generate sound effects in real time, completely procedurally. The plugin offers an interactive interface with 14 parameters and 16 realistic presets (Formula One, monster truck, motorbike…). It also offers various functions to change the parameters of the sound effect at runtime. Hence the parameters can be linked to any in-game events which can lead to very organic and interesting results.

Here’s a demonstration video.

The engine is available as a native Unity audio plugin at https://assetstore.unity.com/packages/tools/audio/nemisindo-engine-procedural-sound-effects-222246

You can also try out the engine model on Nemisindo’s online sound design platform at

https://nemisindo.com/models/engine-advanced.html