Congratulations Dr. Rod Selfridge!

This afternoon one of our PhD student researchers, Rod Selfridge, successfully defended his PhD. The form of these exams, or vivas, varies from country to country, and even institution to institution, which we discussed previously. Here, its pretty gruelling; behind closed doors, with two expert examiners probing every aspect of the PhD.

Rod’s PhD was on ‘Real-time sound synthesis of aeroacoustic sounds using physical models.’ Aeroacoustic sounds are those generated from turbulent fluid motion or aerodynamic forces, like wind whistling or the swoosh of a sword. But when researchers simulate such phenomena, they usually use highly computational approaches. If you need to analyse airplane noise, it might be okay to spend hours of computing time for a few seconds of sound, but you can’t use that approach in games or virtual reality. The alternative is procedural audio, which involves real-time and controllable sound generation. But that is usually not based on the actual physics that generated the sound. For complicated sounds, at best it is inspired by the physics.

Rod wondered if physical models could be implemented in a procedural audio context. For this, he took a fairly novel approach. Physical modelling often involves gridding up a space and looking at the interaction between each grid element , such as in finite difference time domain methods. But there are equations explaining many aspects of aeroacoustics, so why not build them directly into the model. This is like the difference between modelling a bouncing ball by building a dynamic model of the space in which it could move, or you could just apply Newton’s laws of motion. And Rod took the latter approach. Here’s a slick video summarising what the PhD is about,

It worked. He was able to apply real-time, interactive physical models of propeller sounds, aeolian tones, cavity tones, edge tones, the aeolian harp, a bullroarer. He won the Silver Design Award and Best Student Paper Award at the 141st AES Convention, and the Best Paper Award at the Sound and Music Computing conference. And he produced some great demonstration videos of his work, like

and

and

and

Rod also contributed a lot of great blog entries,

So congratulations to Dr. Rod Selfridge, and best of luck with his future endeavours. 🙂

This is the first blog entry I’ve written for a graduating PhD student. I really should do it for all of them- they’ve all been doing great stuff.

And finally, here’s a list of all Rod’s papers as a member of the Intelligent Sound Engineering team.

·        R. Selfridge, D. Moffat, E. Avital and J. D. Reiss, ‘Creating Real-Time Aeroacoustic Sound Effects Using Physically Derived Models,’ Journal of the Audio Engineering Society, 66 (7/8), pp. 594–607, July/August 2018, DOI: https://doi.org/10.17743/jaes.2018.0033

·        R. Selfridge, D. Moffat and J. D. Reiss, ‘Sound Synthesis of Objects Swinging through Air Using Physical Models,’ Applied Sciences, v. 7 (11), Nov. 2017, Online version doi:10.3390/app7111177

·        R. Selfridge, J. D. Reiss, E. Avital, Physically Derived Synthesis Model of an Edge Tone, Audio Engineering Society Convention 144, May 2018

·        R. Selfridge, D. Moffat and J. D. Reiss, ‘Physically Derived Sound Synthesis Model of a Propeller,’ Audio Mostly, London, 2017

·        R. Selfridge, D. Moffat and J. D. Reiss, ‘Physically Derived Synthesis Model of a Cavity Tone,’ Digital Audio Effects (DAFx) Conf., Edinburgh, September 5–9, 2017

·        R. Selfridge, D. J. Moffat and J. D. Reiss, ‘Real-time physical model for synthesis of sword swing sounds,’ Best paper award, Sound and Music Computing (SMC), Helsinki, July 5-8, 2017.

·        R. Selfridge, D. J. Moffat, E. Avital, and J. D. Reiss, ‘Real-time physical model of an Aeolian harp,’ 24th International Congress on Sound and Vibration (ICSV), London, July 23-27, 2017.

·        R. Selfridge, J. D. Reiss, E. Avital, and X. Tang, “Physically derived synthesis model of aeolian tones,” winner of the Best Student Paper award, 141st Audio Engineering Society Convention USA, 2016.

·        R. Selfridge and J. D. Reiss, Interactive Mixing Using the Wii Controller, AES 130th Convention, May 2011.

Advertisements

Aeroacoustic Sound Effects – Journal Article

I am delighted to be able to announce that my article on Creating Real-Time Aeroacoustic Sound Effects Using Physically Informed Models is in this months Journal of the Audio Engineering Society. This is an invited article following winning the best paper award at the Audio Engineering Society 141st Convention in LA. It is an open access article so free for all to download!

The article extends the original paper by examining how the Aeolian tone synthesis models can be used to create a number of sound effects. The benefits of these models are that the produce plausible sound effects which operate in real-time. Users are presented with a number of highly relevant parameters to control the effects which can be mapped directly to 3D models within game engines.

The basics of the Aeolian tone were given in a previous blog post. To summarise, a tone is generated when air passes around an object and vortices are shed behind it. Fluid dynamic equations are available which allow a prediction of the tone frequency based on the physics of the interaction between the air and object. The Aeolian tone is modelled as a compact sound source.

To model a sword or similar object a number of these compact sound sources are placed in a row. A previous blog post describes this in more detail. The majority of compact sound sources are placed at the tip as this is where the airspeed is greatest and the greatest sound is generated.

The behaviour of a sword when being swung has to be modelled which then used to control some of the parameters in the equations. This behaviour can be controlled by a game engine making fully integrated procedural audio models.

The sword model was extended to include objects like a baseball bat and golf club, as well as a broom handle. The compact sound source of a cavity tone was also added in to replicate swords which have grooved profiles. Subjective evaluation gave excellent results, especially for thicker objects which were perceived as plausible as pre-recorded samples.

The synthesis model could be extended to look at a range of sword cross sections as well as any influence of the material of the sword. It is envisaged that other sporting equipment which swing or fly through the air could be modelled using compact sound sources.

A propeller sound is one which is common in games and film and partially based on the sounds generated from the Aeolian tone and vortex shedding. As a blade passes through the air vortices are shed at a specific frequency along the length. To model individual propeller blades the profiles of a number were obtained with specific span length (centre to tip) and chord lengths (leading edge to trailing edge).

Another major sound source is the loading sounds generated by the torque and thrust. A procedure for modelling these sounds is outlined in the article. Missing from the propeller model is distortion sounds. These are more associated with rotors which turn in the horizontal plane.

An important sound when hearing a propeller powered aircraft is the engine sound. The one taken for this model was based on one of Andy Farnell’s from his book Designing Sound. Once complete a user is able to select an aircraft from a pre-programmed bank and set the flight path. If linked to a game engine the physical dimensions and flight paths can all be controlled procedurally.

Listening tests indicate that the synthesis model was as plausible as an alternative method but still not as plausible as pre-recorded samples. It is believed that results may have been more favourable if modelling electric-powered drones and aircraft which do not have the sound of a combustion engine.

The final model exploring the use of the Aeolian tone was that of an Aeolian Harp. This is a musical instrument that is activated by wind blowing around the strings. The vortices that are shed behind the string can activate a mechanical vibration if they are around the frequency of one of the strings natural harmonics. This produces a distinctive sound.

The digital model allows a user to synthesis a harp of up to 13 strings. Tension, mass density, length and diameter can all be adjusted to replicate a wide variety of string material and harp size. Users can also control a wind model modified from one presented in Andy Farnell’s book Designing Sound, with control over the amount of gusts. Listening tests indicate that the sound is not as plausible as pre-recorded ones but is as plausible as alternative synthesis methods.

The article describes the design processes in more detail as well as the fluid dynamic principles each was developed from. All models developed are open source and implemented in pure data. Links to these are in the paper as well as my previous publications. Demo videos can be found on YouTube.

FAST Industry Day, Thursday 25 October, 2 – 8 pm, Abbey Road Studios

FAST: Fusing Audio and Semantic Technologies for Intelligent Music Production and Consumption

Music’s changing fast: FAST is changing music.Showcasing the culmination of five years of digital music research, FAST (Fusing Audio and Semantic Technologies) is hosting an invite onlyindustry day at Abbey Road Studios on Thursday 25 October, 2 – 8 pm.

Presented by Professor Mark Sandler, Director of the Centre for Digital Music at Queen Mary University of London, the event will showcase to artists, journalists and industry professionals the next generation technologies that will shape the music industry – from production to consumption.

Projects on show include MusicLynx – a web AI app for journeys of discovery through the universe of music; Climb! – an intelligent game-based music composition and performance; FXive – a new start-up for sound effect synthesis; tools for interlinking composition and immersive experience, and many more. Plus, Grateful Live – a unique online fanzine for Grateful Dead fans to learn more about the band, the music and their concerts. All the research comes from the labs of 3 of the UK’s top universities:  Queen Mary’s Centre for Digital Music, Nottingham’s Mixed Reality Lab and Oxford’s e-Research Centre.

The programme will consist of an exciting afternoon and evening of talks and demonstrations, and an expert panel discussion.

Please contact Dr. Jasmina Bolfek-Radovani, FAST Programme Manager at fast-impact@qmul.ac.uk if you are interested in attending the event.

Capacity is limited and your place cannot be guaranteed.

For further details about the FAST project, visit: www.semanticaudio.ac.uk

You can also follow us on twitter @semanticaudio

Cross-adaptive audio effects: automatic mixing, live performance and everything in between

Our paper on Applications of cross-adaptive audio effects: automatic mixing, live performance and everything in between has just been published in Frontiers in Digital Humanities. It is a systematic review of cross-adaptive audio effects and their applications.

Cross-adaptive effects extend the boundaries of traditional audio effects by having many inputs and outputs, and deriving their behavior based on analysis of the signals and their interaction. This allows the audio effects to adapt to different material, seemingly being aware of what they do and listening to the signals. Here’s a block diagram showing how a cross-adaptive audio effect modifies a signal.

cross-adaptive architecture

Last year, we published a paper reviewing the history of automatic mixing, almost exactly ten years to the day from when automatic mixing was first extended beyond simple gain changes for speech applications. These automatic mixing applications rely on cross-adaptive effects, but the effects can do so much more.

Here’s an example automatic mixing system from our youtube channel, IntelligentSoundEng.

When a musician uses the signals of other performers directly to inform the timbral character of her own instrument, it enables a radical expansion of interaction during music making. Exploring this was the goal of the Cross-adaptive processing for musical intervention project, led by Oeyvind Brandtsegg, which we discussed in an earlier blog entry. Using cross-adaptive audio effects, musicians can exert control over each the instruments and performance of other musicians, both leading to new competitive aspects and new synergies.

Here’s a short video demonstrating this.

Despite various projects, research and applications involving cross-adaptive audio effects, there is still a fair amount of confusion surrounding the topic. There are multiple definitions, sometimes even by the same authors. So this paper gives a brief history of applications as well as a classification of effects types and clarifies issues that have come up in earlier literature. It further defines the field, lays a formal framework, explores technical aspects and applications, and considers the future from artistic, perceptual, scientific and engineering perspectives.

Check it out!

The edgiest tone yet…

As my PhD is coming to an end and the writing phase is getting more intense, it seemed about time I described the last of the aeroacoustic sounds I have implemented as a sound effect model. May 24th at the 144th Audio Engineering Society Convention in Milan, I will present ‘Physically Derived Synthesis Model of an Edge Tone.’
The edge tone is the sound created when a planar jet of air strikes an edge or wedge. The edge tone is probably most often seen as means of excitation for flue instruments. These instruments are ones like a recorder, piccolo, flute and pipe organ. For example, in a recorder air is blown by the mouth through a mouthpiece into a planar jet and then onto a wedge. The forces generated couple with the tube body of the recorder and a tone based on the dimension of the tube is generated.

 

Mouthpiece of a recorder

 

The edge tone model I have developed is viewed in isolation rather than coupled to a resonator as in the musical instruments example. While researching the edge tone it seemed clear to me that this tone has not had the same attention as the Aeolian tone I have previously modelled (here) but a volume of research and data was available to help understand and develop this model.

How does the edge tone work?

The most important process in generating the edge tone is the set up of a feedback loop from the nozzle exit to the wedge. This is similar to the process that generates the cavity tone which I discussed here. The diagram below will help with the explanation.

 

Illustration of jet of air striking a wedge

 

The air comes out of the nozzle and travels towards the wedge. A jet of air naturally has some instabilities which are magnified as the jet travels and reaches the wedge. At the wedge, vortices are shed on opposite sides of the wedge and an oscillating pressure pulse is generated. The pressure pulse travels back towards the nozzle and re-enforces the instabilities. At the correct frequency (wavelength) a feedback loop is created and a strong discrete tone can be heard.

 

 

To make the edge tone more complicated, if the air speed is varied or the distance between the nozzle exit to the wedge is varies, different modes exist. The values at which the modes change also exhibit hysteresis – the mode changes up and down do not occur at the same airspeed or distance.

Creating a synthesis model

There are a number of equations defined by researchers from the fluid dynamics field, each unique but depend on an integer mode number. Nowhere in my search did I find a method of predicting the mode number. Unlike previous modelling approaches, I decided to collate all the results I had where the mode number was given, both wind tunnel measurements and computational simulations. These were then input to the Weka machine learning workbench and a decision tree was devised. This was then implemented to predict the mode number.

 

All the prediction equations had a significant error compared to the measured and simulated results so again the results were used to create a new equation to predict the frequency for each mode.

 

With the mode predicted and the subsequent frequency predicted, the actual sound synthesis was generated by noise shaping with a white noise source and a bandpass filter. The Q value for the filter was unknown but, as with the cavity tone, it is known that the more turbulent the flow the smaller and more diffuse the vortices and the wider the band of frequencies around the predicted edge tone is. The Q value for the bandpass was set to be proportional to this.

And what next…?

Unlike the Aeolian tone where I was able to create a number of sound effects, the edge tone has not yet been implemented into a wider model. This is due to time rather than anything else. One area of further development which would be of great interest would be to couple the edge tone model to a resonator to emulate a musical instrument. Some previous synthesis models use a white noise source and an excitation or a signal based on the residual between an actual sample and the model of the resonator.

 

Once a standing wave has been established in the resonator, the edge tone locks in at that frequency rather than the one predicted in the equation. So the predicted edge tone may only be present while a musical note is in the transient state but it is known that this has a strong influence over the timbre and may have interesting results.

 

For an analysis of whistles and how their design affects their sound check out his article. The feedback mechanism described for the edge tone also very similar to the one that generates the hole tone. This is the discrete tone that is generated by a boiling kettle. This is usually a circular jet striking a plate with a circular hole and a feedback loop established.

 

Hole tone form a kettle

 

A very similar tone can be generated by a vertical take-off and landing vehicle when the jets from the lift fans are pointing down to the ground or deck. These are both areas for future development and where interesting sound effects could be made.

 

Vertical take-off of a Harrier jet

 

A short history of graphic and parametric equalization

Early equalizers were fixed and integrated into the circuits of audio receivers or phonograph playback systems. The advent of motion picture sound saw the emergence of variable equalization. Notably, John Volkman’s external equalizer design from the 1930s featured a set of selectable frequencies with boosts and cuts, and is sometimes considered to be the first operator-variable equalizer.
Throughout the 1950s and 1960s, equalizers grew in popularity, finding applications in sound post-production and speech enhancement. The Langevin Model EQ-251A, an early program equalizer with slide controls, was a precursor to the graphic equalizer. One slider controlled a bass shelving filter, and the other provided peaking boost/cut with four switchable center frequencies. Each filter had switchable frequencies and used a 15-position slide switch to adjust the gain. Cinema Engineering introduced the first graphic equalizer. It could adjust six bands with a boost or cut range . However, with graphic equalizers, engineers were still limited to the constraints imposed by the number and location of bands.
By 1967, Saul Walker introduced the API 550A equalizer, whose bandwidth is inherently altered relative to the amount of signal boosted. This EQ, like others of its time, featured a fixed selection of frequencies, and variable boost or cut controls at those frequencies. In 1971, Daniel Flickinger invented an important tunable equalizer. His circuit, known as `sweepable EQ’, allowed arbitrary selection of frequency and gain in three overlapping bands.
In 1966, Burgess Macneal and George Massenburg began work on a new recording console. Macneal and Massenburg, who was still a teenager, conceptualized an idea for a sweep-tunable EQ that would avoid inductors and switches. Soon after, Bob Meushaw, a friend of Massenburg, built a three-band, frequency adjustable, fixed-Q equalizer. When asked who invented the parametric equalizer, Massenburg stated “four people could possibly lay claim to the modern concept: Bob Meushaw, Burgess Macneal, Daniel Flickinger, and myself… Our (Bob’s, Burgess’ and my) sweep-tunable EQ was borne, more or less, out of an idea that Burgess and I had around 1966 or 1967 for an EQ… three controls adjusting, independently, the parameters for each of three bands for a recording console… I wrote and delivered the AES paper on Parametrics at the Los Angeles show in 1972… It’s the first mention of `Parametric’ associated with sweep-tunable EQ.”
See
  • Bohn, D.A. Operator adjustable equalizers: An overview. In Proc. Audio Eng. Soc. 6th Int. Conf.: Sound Reinforcement; 1988; pp. 369–381.
  • Reiss, J.D.; McPherson, A. Filter effects (Chapter 4). In Audio Effects: Theory, Implementation and Application; CRC Press: Boca Raton, FL, USA, 2015; pp. 89–124.
  • Flickinger, D. Amplifier system utilizing regenerative and degenerative feedback to shape the frequency response. U.S. Patent #3,752,928 1973.
  • Massenburg, G. Parametric equalization. In Proc. Audio Eng. Soc. 42nd Conv.; 1972.