The edgiest tone yet…

As my PhD is coming to an end and the writing phase is getting more intense, it seemed about time I described the last of the aeroacoustic sounds I have implemented as a sound effect model. The edge tone is the sound created when a planar jet of air strikes an edge or wedge.

The edge tone is probably most often seen as means of excitation for flue instruments. These instruments are ones like a recorder, piccolo, flute and pipe organ. For example, in a recorder air is blown by the mouth through a mouthpiece into a planar jet and then onto a wedge. The forces generated couple with the tube body of the recorder and a tone based on the dimension of the tube is generated.

Mouthpiece of a recorder

The edge tone model I have developed is viewed in isolation rather than coupled to a resonator as in the musical instruments example. While researching the edge tone it seemed clear to me that this tone has not had the same attention as the Aeolian tone I have previously modelled (here) but a volume of research and data was available to help understand and develop this model.

How does the edge tone work?

The most important process in generating the edge tone is the set up of a feedback loop from the nozzle exit to the wedge. This is similar to the process that generates the cavity tone which I discussed here. The diagram below will help with the explanation.

Illustration of jet of air striking a wedge

The air comes out of the nozzle and travels towards the wedge. A jet of air naturally has some instabilities which are magnified as the jet travels and reaches the wedge. At the wedge, vortices are shed on opposite sides of the wedge and an oscillating pressure pulse is generated. The pressure pulse travels back towards the nozzle and re-enforces the instabilities. At the correct frequency (wavelength) a feedback loop is created and a strong discrete tone can be heard.

To make the edge tone more complicated, if the air speed is varied or the distance between the nozzle exit to the wedge is varies, different modes exist. The values at which the modes change also exhibit hysteresis – the mode changes up and down do not occur at the same airspeed or distance.

Creating a synthesis model

There are a number of equations defined by researchers from the fluid dynamics field, each unique but depend on an integer mode number. Nowhere in my search did I find a method of predicting the mode number. Unlike previous modelling approaches, I decided to collate all the results I had where the mode number was given, both wind tunnel measurements and computational simulations. These were then input to the Weka machine learning workbench and a decision tree was devised. This was then implemented to predict the mode number.

All the prediction equations had a significant error compared to the measured and simulated results so again the results were used to create a new equation to predict the frequency for each mode.

With the mode predicted and the subsequent frequency predicted, the actual sound synthesis was generated by noise shaping with a white noise source and a bandpass filter. The Q value for the filter was unknown but, as with the cavity tone, it is known that the more turbulent the flow the smaller and more diffuse the vortices and the wider the band of frequencies around the predicted edge tone is. The Q value for the bandpass was set to be proportional to this.

And what next…?

Unlike the Aeolian tone where I was able to create a number of sound effects, the edge tone has not yet been implemented into a wider model. This is due to time rather than anything else. One area of further development which would be of great interest would be to couple the edge tone model to a resonator to emulate a musical instrument. Some previous synthesis models use a white noise source and an excitation or a signal based on the residual between an actual sample and the model of the resonator.

Once a standing wave has been established in the resonator, the edge tone locks in at that frequency rather than the one predicted in the equation. So the predicted edge tone may only be present while a musical note is in the transient state but it is known that this has a strong influence over the timbre and may have interesting results.

For an analysis of whistles and how their design affects their sound check out his article. The feedback mechanism described for the edge tone also very similar to the one that generates the hole tone. This is the discrete tone that is generated by a boiling kettle. This is usually a circular jet striking a plate with a circular hole and a feedback loop established.

Hole tone form a kettle

A very similar tone can be generated by a vertical take-off and landing vehicle when the jets from the lift fans are pointing down to the ground or deck. These are both areas for future development and where interesting sound effects could be made.

Vertical take-off of a Harrier jet

 

Advertisements

Sound Synthesis – Are we there yet?

TL;DR. Yes

At the beginning of my PhD, I began to read the sound effect synthesis literature, and I quickly discovered that there was little to no standardisation or consistency in evaluation of sound effect synthesis models – particularly in relations to the sounds they produce. Surely one of the most important aspects of a synthetic system, is whether it can artifically produce a convincing replacement for what it is intended to synthesize. We could have the most intractable and relatable sound model in the world, but if it does not sound anything like it is intended to, then will any sound designers or end users ever use it?

There are many different methods for measuring how effective a sound synthesis model is. Jaffe proposed evaluating synthesis techniques for music based on ten criteria. However, only two of the ten criteria actually consider any sounds made by the synthesiser.

This is crazy! How can anyone know what synthesis method can produce a convincingly realistic sound?

So, we performed a formal evaluation study, where a range of different synthesis techniques where compared in a range of different situations. Some synthesis techniques are indistinguishable from a recorded sample, in a fixed medium environment. In short – Yes, we are there yet. There are sound synthesis methods that sound more realistic than high quality recorded samples. But there is clearly so much more work to be done…

For more information read this paper

Creative projects in sound design and audio effects

This past semester I taught two classes (modules), Sound Design and Digital Audio Effects. In both classes, the final assignment involves creating an original work that involves audio programming and using concepts taught in class. But the students also have a lot of free reign to experiment and explore their own ideas.

The results are always great. Lots of really cool ideas, many of which could lead to a publication, or would be great to listen to regardless of the fact that it was an assignment. Here’s a few examples.

From the Sound Design class;

  • Synthesizing THX’s audio trademark, Deep Note. This is a complex sound, ‘a distinctive synthesized crescendo that glissandos from a low rumble to a high pitch’. It was created by the legendary James Moorer, who is responsible for some of the greatest papers ever published in the Journal of the Audio Engineering Society.
  • Recreating the sound of a Space Shuttle launch, with separate components for ‘Air Burning/Lapping’ and ‘Flame Eruption/Flame Exposing’ by generating the sounds of the Combustion chain and the Exhaust chain.
  • A student created a soundscape inspired by the 1968 Romanian play ‘Jonah (A four scenes tragedy)’,  written by Marin Sorescu. Published in 1968, when Romania was ruled by the communist regime. By carefully modulating the volume of filtered noise, she was able to achieve some great synthesis of waves crashing on a shore.
  • One student made a great drum and bass track, manipulating samples and mixing in some of his own recorded sounds. These included a nice ‘thud’ by filtering the sound of a tightened towel, percussive sounds by shaking rice in a plastic container. and the sizzling sound of frying bacon for tape hiss.
  • Synthesizing the sound of a motorbike, including engine startup, gears and driving sound, gear lever click and indicator.
  • A short audio piece to accompany a ghost story, using synthesised and recorded sounds. What I really like is that the student storyboarded it.

storyboard

  • A train on a stormy day, which had the neat trick of converting a footstep synthesis model into the chugging of a train.
  • The sounds of the London Underground, doors sliding and beeping, bumps and breaks… all fully synthesized.

And from the Digital Audio Effects class;

  • An autotune specifically for bass guitar. We discussed auto-tune and its unusual history previously.
  • Sound wave propagation causes temperature variation, but speed of sound is a function of temperature. Notably, the positive half cycle of a wave (compression) causes an increase in temperature and velocity, while the negative half (rarefaction) causes a decrease in temperature and velocity, turning a sine wave into something like a sawtooth. This effect is only significant in high pressure sound waves. Its also frequency dependent; high frequency components travel faster than low frequency components.
    Mark Daunt created a MIDI instrument as a VST Plug-in that generates sounds based on this shock-wave formation formula. Sliders allow the user to adjust parameters in the formula and use a MIDI keyboard to play tones that express characteristics of the calculated waveforms.

  • Synthesizing applause, a subject which we have discussed here before. The student has been working in this area for another project, but made significant improvements for the assignment, including adding presets for various conditions.
  • A student devised a distortion effect based on waveshaping in the form of a weighted sum of Legendre polynomials. These are interesting functions and her resulting sounds are surprising and pleasing. Its the type of work that could be taken a lot further.
  • One student had a bug in an implementation of a filter. Noticing that it created some interesting sounds, he managed to turn it into a cool original distortion effect.
  • There’s an Octagon-shaped room with strange acoustics here on campus. Using a database of impulse response measurements from the room, one student created a VST plug-in that allows the user to hear how audio sounds for any source and microphone positions. In earlier blog entries, we discussed related topics, acoustic reverberators and anechoic chambers.

Screen Shot 2018-03-22 at 20.21.58-14

  • Another excellent sounding audio effect was a spectral delay using the phase vocoder, with delays applied differently depending on frequency bin. This created a sound like ‘stars falling from the sky’. Here’s a sine sweep before and after the effect is applied.

https://soundcloud.com/justjosh71/sine-sweep-original

There were many other interesting assignments (plucked string effect for piano synthesizer, enhanced chorus effects, inharmonic resonator, an all-in-one plug-in to recreate 80s rock/pop guitar effects…). But this selection really shows both the talent of the students and the possibilities to create new and interesting sounds.

Audio Engineering Society E-library

I try to avoid too much promotion in this blog, but in this case I think its justified. I’m involved in advancing a resource from a non-profit professional organisation, the Audio Engineering Society. They do lots and lots of different things, promoting the science, education and practice of all things audio engineering related. Among others, they’ve been publishing research in the area for almost 70 years, and institutions can get full access to all the content in a searchable library. In recent posts, I’ve written about some of the greatest papers ever published there, Part 1 and Part 2, and about one of my own contributions.

In an ideal world, this would all be Open Access . But publishing still costs money, so the AES support both gold Open Access (free to all, but authors pay Article Processing Charges) and the traditional model, where its free to publish but individuals or institutions subscribe or articles can be purchased individually. AES members get free access. I could write many blog articles just about Open Access (should I?)- its never as straightforward as it seems. At its best it is freely disseminating information for the benefit of all, but at its worst its like Pay to Play, a highly criticised practice for the music industry, and gives publishers an incentive to lower acceptance standards. But for now I’ll just point out that the AES does its absolute best to keep the costs down, regardless of publishing model, and the costs are generally much less than similar publishers.

Anyway, the AES realised that one of the most cost effective ways to get our content out to large communities is through institutional licenses or subscriptions. And we’re missing an opportunity here since we haven’t really promoted this option. And everybody benefits from it; wider dissemination of knowledge and research, more awareness of the AES, better access, etc. With this in mind, the AES issued the following press release, which I have copied verbatim. You can also find it as a tweet, blog entry or facebook post.

AES_ELibrary

AES E-Library Subscriptions Benefit Institutions and Organizations

— The Audio Engineering Society E-Library is the world’s largest collection of audio industry resources, and subscriptions provide access to extensive content for research, product development and education — 

New York, NY, March 22, 2018 — Does your research staff, faculty or students deserve access to the world’s most comprehensive collection of audio information? The continuously growing Audio Engineering Society (AES) E-Library contains over 16,000 fully searchable PDF files documenting the progression of audio research from 1953 to the present day. It includes every AES paper published from every AES convention and conference, as well as those published in the Journal of the Audio Engineering Society. From the phonograph to MP3s, from early concepts of digital audio through its fulfillment as the mainstay of audio production, distribution and reproduction, to leading-edge realization of spatial audio and audio for augmented and virtual reality, the E-Library provides a gateway to both the historical and the forward-looking foundational knowledge that sustains an entire industry.  

The AES E-Library has become the go-to online resource for anyone looking to gain instant access to the vast amount of information gathered by the Audio Engineering Society through research, presentations, interviews, conventions, section meetings and more. “Our academic and research staff, and PhD and undergraduate Tonmeister students, use the AES E-Library a lot,” says Dr. Tim Brookes, Senior Lecturer in Audio & Director of Research Institute of Sound Recording (IoSR) University of Surrey. “It’s an invaluable resource for our teaching, for independent student study and, of course, for our research.” 

“Researchers, academics and students benefit from E-Library access daily,” says Joshua Reiss, Chair of the AES Publications Policy Committee, “while many relevant institutions – academic, governmental or corporate – do not have an institutional license of the AES E-library, which means their staff or students are missing out on all the wonderful content there. We encourage all involved in audio research and investigation to inquire if their libraries have an E-Library subscription and, if not, suggest the library subscribe.” 

E-Library subscriptions can be obtained directly from the AES or through journal bundling services. A subscription allows a library’s users to download any document in the E-Library at no additional cost. 

“As an international audio company with over 25,000 employees world-wide, the AES E-library has been an incredibly valuable resource used by Harman audio researchers, engineers, patent lawyers and others,” says Dr. Sean Olive, Acoustic Research Fellow, Harman International. “It has paid for itself many times over.” 

The fee for an institutional online E-Library subscription is $1800 per year, which is significantly less than equivalent publisher licenses. 

To search the E-library, go to http://www.aes.org/e-lib/

To arrange for an institutional license, contact Lori Jackson directly at lori.jackson@aes.org, or go to http://www.aes.org/e-lib/subscribe/.

 

About the Audio Engineering Society
The Audio Engineering Society, celebrating its 70th anniversary in 2018, now counts over 12,000 members throughout the U.S., Latin America, Europe, Japan and the Far East. The organization serves as the pivotal force in the exchange and dissemination of technical information for the industry. Currently, its members are affiliated with 90 AES professional sections and more than 120 AES student sections around the world. Section activities include guest speakers, technical tours, demonstrations and social functions. Through local AES section events, members experience valuable opportunities for professional networking and personal growth. For additional information visit http://www.aes.org.

Join the conversation and keep up with the latest AES News and Events:
Twitter: #AESorg (AES Official) 
Facebook: http://facebook.com/AES.org

You’re invited to my Inaugural Lecture as Professor of Audio Engineering

When writing a blog like this, there’s sometimes a thin line between information and promotion. I suppose this one is on the promotion-side, but its a big deal for me and for at least a few of you it will involve some interesting information.

Queen Mary University of London has Inaugural Lectures for all its professors. I was promoted to Professor last year and so its time for me to do mine. It will be an evening lecture at the University on April 17th, free for all and open to the public. Its quite an event, with a large crowd and a reception after the talk.

I’m going to try to tie together a lot of strands of my research (when I say ‘my’, I mean the great research done by my students, staff and collaborators). That won’t be too hard since there are some common themes throughout. But I’m also going to try to make it as fun and engaging as possible, lots of demonstrations, no dense formal PowerPoint, and a bit of theatre.

You can register online by going to https://www.eventbrite.co.uk/e/do-you-hear-what-i-hear-the-science-of-everyday-sounds-tickets-43749224107 and it has all the information about time, location and so on.

Here’s the full details.

Do you hear what I hear? The science of everyday sounds.

The Inaugural Lecture of Professor Josh Reiss, Professor of Audio Engineering

Tue 17 April 2018, 18:30 – 19:30 BST
ArtsTwo, Queen Mary Mile End Campus
327 Mile End Road, London, E1 4NS

item178407

Details: The sounds around us shape our perception of the world. In films, games, music and virtual reality, we recreate those sounds or create unreal sounds to evoke emotions and capture the imagination. But there is a world of fascinating phenomena related to sound and perception that is not yet understood. If we can gain a deep understanding of how we perceive and respond to complex audio, we could not only interpret the produced content, but we could create new content of unprecedented quality and range.
This talk considers the possibilities opened up by such research. What are the limits of human hearing? Can we create a realistic virtual world without relying on recorded samples? If every sound in a major film or game soundtrack were computer-generated, could we reach a level of realism comparable to modern computer graphics? Could a robot replace the sound engineer? Investigating such questions leads to a deeper understanding of auditory perception, and has the potential to revolutionise sound design and music production. Research breakthroughs concerning such questions will be discussed, and cutting-edge technologies will be demonstrated.

Biography: Josh Reiss is a Professor of Audio Engineering with the Centre for Digital Music at Queen Mary University of London. He has published more than 200 scientific papers (including over 50 in premier journals and 4 best paper awards), and co-authored the textbook Audio Effects: Theory, Implementation and Application. His research has been featured in dozens of original articles and interviews since 2007, including Scientific American, New Scientist, Guardian, Forbes magazine, La Presse and on BBC Radio 4, BBC World Service, Channel 4, Radio Deutsche Welle, LBC and ITN, among others. He is a former Governor of the Audio Engineering Society (AES), chair of their Publications Policy Committee, and co-chair of the Technical Committee on High-resolution Audio. His Royal Academy of Engineering Enterprise Fellowship resulted in founding the high-tech spin-out company, LandR, which currently has over a million and a half subscribers and is valued at over £30M. He has investigated psychoacoustics, sound synthesis, multichannel signal processing, intelligent music production, and digital audio effects. His primary focus of research, which ties together many of the above topics, is on the use of state-of-the-art signal processing techniques for professional sound engineering. He maintains a popular blog, YouTube channel and twitter feed for scientific education and dissemination of research activities.

 

Greatest JAES papers of all time, Part 2

Last week I revealed Part 1 of the greatest ever papers published in the Journal of the Audio Engineering Society (JAES). JAES is the premier peer-reviewed journal devoted exclusively to audio technology, and the flagship publication of the AES. This week, its time for Part 2. There’s little rhyme or reason to how I divided up and selected the papers, other than I started by looking at the most highly cited ones according to Google Scholar. But all the papers listed here have had major impact on the science, education and practice of audio engineering and related fields.

All of the papers below are available from the Audio Engineering Society (AES) E-library, the world’s most comprehensive collection of audio information. It contains over 16,000 fully searchable PDF files documenting the progression of audio research from 1953 to the present day. It includes every AES paper published at a convention, conference or in the Journal. Members of the AES get free access to the E-library. To arrange for an institutional license, giving full access to all members of an institution, contact Lori Jackson Lori Jackson directly, or go to http://www.aes.org/e-lib/subscribe/ .

And without further ado, here are the rest of the Selected greatest JAES papers

More than any other work, this 1992 paper by Stanley Lipshitz and co-authors has resulted in the correct application of dither by music production. Its one possible reason that digital recording quality improved after the early years of the Compact Disc (though the loudness wars reversed that trend). As renowned mastering engineer Bob Katz put it, “if you want to get your digital audio done just right, then you should learn about dither,” and there is no better resource than this paper.

According to Wikipedia, this 1993 paper coined the term Auralization as an analogy to visualization for rendering audible (imaginary) sound fields. This general research area of understanding and rendering the sound field of acoustic spaces has resulted in several other highly influential papers. Berkhout’s 1988 A holographic approach to acoustic control (575 citations) described the appealingly named acoustic holography method for rendering sound fields. In 1999, the groundbreaking Creating interactive virtual acoustic environments (427 citations) took this further, laying out the theory and challenges of virtual acoustics rendering, and paving the way for highly realistic audio in today’s Virtual Reality systems.

The Schroeder reverberator was first described here, way back in 1962. It has become the basis for almost all algorithmic reverberation approaches. Manfred Schroeder was another great innovator in the audio engineering field. A long transcript of a fascinating interview is available here, and a short video interview below.

These two famous papers are the basis for the Thiele Small parameters. Thiele rigorously analysed and simulated the performance of loudspeakers in the first paper from 1971, and Small greatly extended the work in the second paper in 1972. Both had initially published the work in small Australian journals, but it didn’t get widely recognised until the JAES publications. These equations form the basis for much of loudspeaker design.

Check out;

or the dozens of youtube videos about choosing and designing loudspeakers which make use of these parameters.

This is the first English language publication to describe the Haas effect, named after the author. Also called the precedence effect, it investigated the phenomenon that when sending the same signal to two loudspeakers, a small delay between the speakers results in the sound appearing to come just from one speaker. Its now widely used in sound reinforcement systems, and in audio production to give a sense of depth or more realistic panning (the Haas trick).

Hass-effect

This is the first ever research paper published in JAES. Published in August 1949, it set a high standard for rigour, while at the same time emphasising that many publications will have strong relevance not just to researchers, but to audiophiles and practitioners as well.

It described a new instrument for frequency response measurement and display. People just love impulse response and transfer function measurements, and some of the most highly cited JAES papers are on this topic; 1983’s An efficient algorithm for measuring the impulse response using pseudorandom noise (308 citations), Transfer-function measurement with maximum-length sequences (771 citations), the 2001 paper from a Brazil-based team, Transfer-function measurement with sweeps (722 citations), and finally Comparison of different impulse response measurement techniques (276 citations) in 2002. With a direct link between theory and new applications, these papers on maximum length sequence approaches and sine sweeps were major advances over the alternatives, and changed the way such measurements are made.

And the winner is… Ville Pulkki’s Vector Base Amplitude Panning (VBAP) paper! This is the highest cited paper in JAES. Besides deriving the stereo panning law from basic geometry, it unveiled VBAP, an intuitive and now widely used spatial audio technique. Ten years later, Pulkki unveiled another groundbreaking spatial audio format, DirAC, in Spatial sound reproduction with directional audio coding (386 citations).

Greatest JAES papers of all time, Part 1

The Journal of the Audio Engineering Society (JAES) is the premier publication of the AES, and is the only peer-reviewed journal devoted exclusively to audio technology. The first issue was published in 1949, though volume 1 began in 1953. For the past 70 years, it has had major impact on the science, education and practice of audio engineering and related fields.

I was curious which were the most important JAES papers, so had a look at Google Scholar to see which had the most citations. This has lots of issues, not just because Scholar won’t find everything, but because a lot of the impact is in products and practice, which doesn’t usually lead to citing the papers. Nevertheless, I looked over the list, picked out some of the most interesting ones and following no rules except my own biases, selected the Greatest Papers of All Time Published in the Journal of the Audio Engineering Society. Not surprisingly, the list is much longer than a single blog entry, so this is just part 1.

All of the papers below are available from the Audio Engineering Society (AES) E-library, the world’s most comprehensive collection of audio information. It contains over 16,000 fully searchable PDF files documenting the progression of audio research from 1953 to the present day. It includes every AES paper published at a convention, conference or in the Journal. Members of the AES get free access to the E-library. To arrange for an institutional license, giving full access to all members of an institution, contact Lori Jackson Lori Jackson directly, or go to http://www.aes.org/e-lib/subscribe/ .

Selected greatest JAES papers

ambisonicsThis is the main ambisonics paper by its originator, Michael Gerzon, and perhaps the first place the theory was described in detail (and very clearly too). Ambisonics is incredibly flexible and elegant. It is now used in a lot of games and has become the preferred audio format for virtual reality. Two other JAES ambisonics papers are also very highly cited. In 1985, Michael Gerzon’s Ambisonics in multichannel broadcasting and video (368 citations) described the high potential of ambisonics for broadcast audio, which is now reaching its potential due to the emergence of object-based audio production. And 2005 saw Mark Poletti’s Three-dimensional surround sound systems based on spherical harmonics (348 citations), which rigorously laid out and generalised all the mathematical theory of ambisonics.

James Moorer

This isn’t one of the highest cited papers, but it still had huge impact, and James Moorer is a legend in the field of audio engineering (see his prescient ‘Audio in the New Millenium‘). The paper popularised the phase vocoder, now one of the most important building blocks of modern audio effects. Auto-tune, anyone?

Richard Heyser’s Time Delay Spectrometry technique allowed one to make high quality anechoic spectral measurements in the presence of a reverberant environment. It was ahead of its time since despite the efficiency and elegance, computing power was not up to employing the method. But by the 1980s, it was possible to perform complex on-site measurements of systems and spaces using Time Delay Spectrometry. The AES now organises Heyser Memorial Lectures in his honor.

hrtf

Together, these two papers by Henrik Møller et al completed transformed the world of binaural audio. The first paper described the first major dataset of detailed HRTFs, and how they vary from subject to subject. The second studied localization performance when subjects listened to a soundfield, the same soundfield using binaural recordings with their own HRTFs, and those soundfields using the HRTFs of others. It nailed down the state of the art and the challenges for future research.

The early MPEG audio standards. MPEG 1 unveiled the MP3, followed by the improved MPEG2 AAC. They changed the face of not just audio encoding, but completely revolutionised music consumption and the music industry.

John Chowning was a pioneer and visionary in computer music. This seminal work described FM synthesis, where the timbre of a simple waveform is changed by frequency modulating it with another frequency also in the audio range, resulting in a surprisingly rich control of audio spectra and their evolution in time. In 1971, Chowning also published The simulation of moving sound sources (278 citations), perhaps the first system (and using digital technology) for synthesising an evolving sound scene.

The famous Glasberg and Moore loudness model is perhaps the most widely used auditory model for loudness and masking estimation. Other aspects of it have appeared in other papers (including A model of loudness applicable to time-varying sounds, 487 citations, 2002).

More greatest papers in the next blog entry.