Sound Talking at the Science Museum featured assorted speakers on sonic semantics

sound-talking-logo-large

On Friday 3 November, Dr Brecht De Man (Centre for Digital Music, Queen Mary University of London) and Dr Melissa Dickson (Diseases of Modern Life, University of Oxford) organised a one-day workshop at the London Science Museum on the topic of language describing sound, and sound emulating language. We discussed it in a previous blog entry, but now we can wrap up and discuss what happened.

Titled ‘Sound Talking‘, it brought together a diverse lineup of speakers around the common theme of sonic semantics. And with diverse we truly mean that: the programme featured a neuroscientist, a historian, an acoustician, and a Grammy-winning sound engineer, among others.

The event was born from a friendship between two academics who had for a while assumed their work could not be more different, with music technology and history of Victorian literature as their respective fields. When learning their topics were both about sound-related language, they set out to find more researchers from maximally different disciplines and make it a day of engaging talks.

After having Dr Dickson as a resident researcher earlier this year, the Science Museum generously hosted the event, providing a very appropriate and ‘neutral’ central London venue. The venue was further supported by the Diseases of Modern Life project, funded by the European Research Council, and the Centre for Digital Music at Queen Mary University of London.

The programme featured (in order of appearance):

  • Maria Chait, Professor of auditory cognitive neuroscience at UCL, on the auditory system as the brain’s early warning system
  • Jonathan Andrews, Reader in the history of psychiatry at Newcastle University, on the soundscape of the Bethlehem Hospital for Lunatics (‘Bedlam’)
  • Melissa Dickson, postdoctoral researcher in Victorian literature at University of Oxford, on the invention of the stethoscope and the development of an associated vocabulary
  • Mariana Lopez, Lecturer in sound production and post production at University of York, on making film accessible for visually impaired audiences through sound design
  • David M. Howard, Professor of Electronic Engineering at Royal Holloway University of London, on the sound of voice and the voice of sound
  • Brecht De Man, postdoctoral researcher in audio engineering at Queen Mary University of London, on defining the language of music production
  • Mandy Parnell, mastering engineer at Black Saloon Studios, on the various languages of artistic direction
  • Trevor Cox, Professor of acoustic engineering at University of Salford, on categorisation of everyday sounds

In addition to this stellar speaker lineup, Aleks Kolkowski (Recording Angels) exhibited an array of historic sound making objects, including tuning forks, listening tubes, a monochord, and a live recording of a wax cylinder. The workshop took place in a museum, after all, where Dr Kolkowski has held a research associateship, so the display was very fitting.

The full program can be found on the event’s web page. Video proceedings of the event are forthcoming.

Advertisements

Your phd examination – the best defense is a good offense

Previously, I’ve written a few blog entries giving research advice, like ‘So you want to write a research paper‘ and ‘What a PhD thesis is really about… really!‘ I thought I’d come up with a good title for this blog entry, but then I saw this.

thesis_defense

The PhD examination is certainly one of the most important moments in a researcher’s career. Its structure differs from country to country, institution to institution, and subject to subject. In some places, the PhD examination is open to the public, and failure is very rare. The student wouldn’t get to that stage unless the committee was confident that only minor issues remained. It might even be a bit of an event, with the committee wearing gowns and some of the student’s family attending.

But in most countries and most subjects it’s a bit more adversarial and passing is not guaranteed. It usually has a small committee. A public talk might be given, but the question and answer sessions are just the student and the committee.

There are lots and lots of guidance online about how to prepare for a PhD exam, and I’m not going to try to summarise them. Instead, I’ll give you some insights from my own experience, being examined, preparing others for a phd examination, or doing the examination myself. And having had experience with students who ranged from near flawless to, unfortunately, almost hopeless.

First off, congratulations for getting to this stage. That is already a major achievement. And keep in mind is that ultimately, it’s the document itself that is most important. If your thesis is strong, and you can explain it and discuss it well, then you’re already in a good position for the defense.

phd031905s

I’ve noticed that there are questions which seem relevant for me to ask in most PhD examinations, and other examiners tend to ask similar ones. So you can certainly prepare for them. The first are the sort of general PhD study questions; what’s it all about? Here’s a few typical ones.

  • Summarise your key findings?
  • What is your main contribution?
  • What is novel/significant/new?
  • What is the impact? your contribution to the field?
  • What are the weakest parts of your thesis?
  • Knowing what you know now, what would you change?

If there were aspects of your PhD study that were unusual, they also might ask you just to clarify things. For instance, I once examined a PhD whose research had taken a very long time. I wanted to know if there was research that hadn’t made it into the thesis, or whether there were technical issues that made the research more challenging. So I asked a question something like, ‘When did you start your phd research? Were there technical reasons it took so long?’ As it turned out, it was due to a perfectly understandable change of supervisor.

And the examiners will want to know what you know about your subject area and the state of the art.

  • Who else is doing research in this subject?
  • What are the most significant results in the last few years?
  • How does your approach differ from others?
  • Please characterise and summarise other approaches to your topic.

Then there will be some questions specific to your field. These questions might touch on the examiners’ knowledge, or on specific aspects of the literature that may or may not have been mentioned in the thesis.

  • Explain, in your own words, the following concepts -.
  • Compare the – and -. What are the fundamental differences?
  • Is all of your work relevant to other — challenges?
  • Why use —? Are there other approaches?
  • How does your work connect to — and — research?

And many examiners will want to know about the impact of the research so far, e.g. publications or demonstrators. If you do have any demonstrations (audio samples, videos, software, interfaces), it’s a good idea to present them, or at least be ready to present them.

  • Are the community aware of your work? Are people using your software?
  • Do you have any publications?
  • Which (other) results could you publish, and where?
  • Have you attended or presented at any conferences? What did you learn from them?

Ct_XEoLXgAAniid

Then typically, the examiners start diving into the fine details of the thesis. So you should know where to find anything in your own document. Its also a good idea to reread your whole document a couple of days before the examination, so that its all fresh in your mind. It could have been a long time since you wrote it!

Spider-viva-1

And best of luck to you!

 

My favorite sessions from the 143rd AES Convention

AES_NY

Recently, several researchers from the audio engineering research team here attended the 143rd Audio Engineering Society Convention in New York. Before the Convention, I wrote a blog entry highlighting a lot of the more interesting or adventurous research that was being presented there. As is usually the case at these Conventions, I have so many meetings to attend that I miss out on a lot of highlights, even ones that I flag up beforehand as ‘must see’. Still, I managed to attend some real gems this time, and I’ll discuss a few of them here.

I’m glad that I attended ‘Audio Engineering with Hearing Loss—A Practical Symposium’ . Hearing loss amongst musicians, audiophiles and audio engineers is an important topic that needs more attention. Overexposure, both prolonged and too loud, is a major cause of hearing dage. In addition to all the issues it causes for anybody, for those in the industry, it affects their ability to work or even appreciate their passion. The session had lots of interesting advice.

The most interesting presentation in the session was from Richard Einhorn, a composer and music producer. In 2010, he lost much of his hearing due to a virus. He woke up one day to find that he had completely lost hearing in his right ear, a condition known as Idiopathic Sudden Sensorineural Hearing Loss. This then evolved into hyperacusis, with extreme distortion, excessive volume and speech intelligibility. In many ways, deafness in the right ear would have been preferred. On top of that, his left ear suffered otosclerosis, where everything was at greatly reduced volume. And given that this was his only functioning ear, the risk of surgery to correct it was too great.

Richard has found some wonderful ways to still function, and even continue working in audio and music, with the limited hearing he still has. There’s a wonderful description of them in Hearing Loss Magazine, and they include the use of the ‘Companion Mic,’ which allowed him to hear from many different locations around a busy, noisy environment, like a crowded restaurant.

Thomas Lund presented ‘The Bandwidth of Human Perception and its Implications for Pro Audio.’ I really wasn’t sure about this before the Convention. I had read the abstract, and thought it might be some meandering, somewhat philosophical talk about hearing perception, with plenty of speculation but lacking in substance. I was very glad to be proven wrong! It had aspects of all of that, but in a very positive sense. It was quite rigorous, essentially a systematic review of research in the field that had been published in medical journals. It looks at the question of auditory perceptual bandwidth, where bandwidth is in a general information theoretic and cognitive sense, not specifically frequency range. The research revolves around the fact that, though we receive many megabits of sensory information every second, it seems that we only use dozens of bits per second of information in our higher level perception. This has lots of implications for listening test design, notably on how to deal with aspects like sample duration or training of participants. This was probably the most fascinating technical talk I saw at the Convention.

There were two papers that I had flagged up as having the most interesting titles, ‘Influence of Audience Noises on the Classical Music Perception on the Example of Anti-cough Candies Unwrapping Noise’, and ‘Acoustic Levitation—Standing Wave Demonstration.’ I had an interesting chat with an author of the first one, Adam Pilch. When walking around much later looking for the poster for the second one, I bump into Adam again. Turns out, he was a co-author on both of them! It looks like Adam Pilch and Bartlomiej Chojnacki (the shared authors on those papers) and their co-authors have an appreciation of the joy of doing research for fun and curiousity, and an appreciation for a good paper title.

Leslie Ann Jones was the Heyser lecturer. The Heyser lecture, named after Richard C. Heyser, is an evening talk given by an eminent individual in audio engineering or related fields. Leslie has had a fascinating career, and gave a talk that makes one realise just how much the industry is changing and growing, and how important are the individuals and opportunities that one encounters in a career.

The last session I attended was also one of the best. Chris Pike, who recently became leader of the audio research team at BBC R&D (he has big shoes to fill, but fits them well and is already racing ahead), presented ‘What’s This? Doctor Who with Spatial Audio!’ . I knew this was going to be good because it involved two of my favorite things, but it was much better than that. The audience were all handed headphones so that they could listen to binaural renderings used throughout the presentation. I love props at technical talks! I also expected the talk to focus almost completely on the binaural, 3d sound rendering for a recent episode, but it was so much more than that. There was quite detailed discussion of audio innovation throughout the more than 50 years of Doctor Who, some of which we have discussed when mentioning Daphne Oram and Delia Derbyshire in our blog entry on female pioneers in audio engineering.

There’s a nice short interview with Chris and colleagues Darran Clement (sound mixer) and Catherine Robinson (audio supervisor) about the binaural sound in Doctor Who on BBC R&D’s blog, and here’s a youtube video promoting the binaural sound in the recent episode;

 

Applied Science Journal Article

We are delighted to announce the publication of our article titled, Sound Synthesis of Objects Swinging through Air Using Physical Models in the Applied Science Special Issue on Sound and Music Computing.

 

The Journal is a revised and extended version of our paper which won a best paper award at the 14th Sound and Music Computing Conference which was held in Espoo, Finland in July 2017. The initial paper presented a physically derived synthesis model used to replicate the sound of sword swings using equations obtained from fluid dynamics, which we discussed in a previous blog entry. In the article we extend listening tests to include sound effects of metal swords, wooden swords, golf clubs, baseball bats and broom handles as well as adding in a cavity tone synthesis model to replicate grooves in the sword profiles. Further test were carried out to see if participants could identify which object our model was replicating by swinging a Wii Controller.
The properties exposed by the sound effects model could be automatically adjusted by a physics engine giving a wide corpus of sounds from one simple model, all based on fundamental fluid dynamics principles. An example of the sword sound linked to the Unity game engine is shown in this video.
 

 

Abstract:
A real-time physically-derived sound synthesis model is presented that replicates the sounds generated as an object swings through the air. Equations obtained from fluid dynamics are used to determine the sounds generated while exposing practical parameters for a user or game engine to vary. Listening tests reveal that for the majority of objects modelled, participants rated the sounds from our model as plausible as actual recordings. The sword sound effect performed worse than others, and it is speculated that one cause may be linked to the difference between expectations of a sound and the actual sound for a given object.
The Applied Science journal is open access and a copy of our article can be downloaded here.

The Audiovisual bounce-inducing effect (Bounce, bounce, bounce… Part II)

Last week we talked about bouncing sounds. Its very much a physical phenomenon, but a lot has been made of a perceptual effect sometimes referred to as the ‘Audiovisual bounce-inducing effect.’ The idea is that if someone is presented with two identical objects moving on a screen in opposing direction and crossing paths, they appear to do just that- cross paths. But if a short sound is played at the moment they first intersect, they appear to bounce off each other.

I’ve read a couple of papers on this, and browsed a few more, and I’ve yet to see anything interesting here.

Consider the figures below. On the left are the two paths taken by the two objects, one with short dashes in blue, one with long dashes in red. Since they are identical (usually just circles on a computer screen), it could just as easily be the paths shown on the right.

AVbounceillusion

So which one is perceived? Well, two common occurrences are;

– Two objects, and one of them passes behind the other. This usually doesn’t produce a sound.
– Two objects, and they bounce off each other, producing the sound of a bounce.

If you show the objects without a sound, it perfectly matches the first scenario. It would be highly unlikely to perceive this as a bounce since then we would expect to hear something. On the other hand, if you play a short sound at the moment the two objects interact, even if it doesn’t exactly match a ‘bounce sound’, it is still a noise at the moment of visual contact. And so this is much more likely to be perceived as a bounce (which clearly produces a sound) than as passing by (which doesn’t). Further studies showed that the more ‘bounce-like’ the sound is, the more likely it is to be perceived as a bounce, and its less likely to be perceived as a bounce if similar sounds are also played when the objects do not intersect.

The literature gives all sorts of fanciful explanations for the basic phenomenon. And maybe someone can enlighten me as to why this is interesting. I suppose, if one begins with the assumption that auditory cues (even silence) do not play a role in perception of motion, then this may be surprising. But to me, this just seems to match everyday experience of sight and sound, and is intuitively obvious.

I should also note that in one of the papers on the ‘Audiovisual bounce-inducing effect’ (Watanabe 2001), the authors committed the cardinal sin of including one of the authors as a test subject and performing standard statistical analysis on the results. There are situations when this sort of thing may be acceptable or even appropriate*, but in which case one should be very careful to take that into account in any analysis and interpretation of results.

* In the following two papers, participants rated multrack audio mixes, where one of the mixes had been created by the participant. But this was intentional, to see whether the participant would rate their own mix highly.

And here’s just a few references on the audiovisual bounce inducing effect.

Grassi M, Casco C. Audiovisual bounce-inducing effect: When sound congruence affects grouping in vision. Attention, Perception, & Psychophysics. 2010 Feb 1;72(2):378-86.

Remijn GB, Ito H, Nakajima Y. Audiovisual integration: An investigation of the ‘streaming-bouncing’phenomenon. Journal of physiological anthropology and applied human science. 2004;23(6):243-7.

Watanabe K, Shimojo S. When sound affects vision: effects of auditory grouping on visual motion perception. Psychological Science. 2001 Mar;12(2):109-16.

Zeljko M, Grove PM. Sensitivity and Bias in the Resolution of Stream-Bounce Stimuli. Perception. 2017 Feb;46(2):178-204.

Bounce, bounce, bounce . . .

bounce

Another in our continuing exploration of everyday sounds (Screams, Applause, Pouring water) is the bouncing ball. It’s a nice one for a blog entry since there are only a small number of papers focused on bouncing, which means we can give a good overview of the field. It’s also one of those sounds that we can identify very clearly; we all know it when we hear it. It has two components that can be treated separately; the sound of a single bounce and the timing between bounces.

Let’s consider the second aspect. If we drop a ball from a certain height and ignore any drag, the time it takes to hit the ground is completely determined by gravity. When it hits the ground, some energy is absorbed on impact. And so it may be traveling downwards with a velocity v1 just before impact, and after impact travels upwards with velocity v2. The ratio v2/v1 is called the coefficient of restitution (COR). A high COR means that the ball travels back up almost to its original height, and a low COR means that most energy is absorbed and it only travels up a short distance.

Knowing COR, one can use simple equations of motion to determine the time between each bounce. And since the sum of the times between bounces is a convergent series, one can find the maximum time until it stops bouncing. Conversely, measuring the coefficient of friction from times between bounces is literally a tabletop physics experiment (Aguiar 2003, Farkas 2006, Schwarz 2013). And kinetic energy depends on the square of the velocity, so we know how much energy is lost with each bounce, which also gives an idea of how the sound levels of successive bounces should decrease.

[The derivation of all this has been left to the reader 😊. But again, its straightforward application of the equations of motion that give time dependence of position and velocity under constant acceleration]

Its not that hard to extend this approach, for instance by including air drag or sloped surfaces. But if you put the ball on a vibrating platform, all sorts of wonderful nonlinear behaviour can be observed; chaos, locking and chattering (Luck 1993).

For instance, have a look at the following video; which shows some interesting behaviour where bouncing balls all seem to organise onto one side of a partition.

So much for the timing of bounces, but what about the sound of a single bounce? Well, Nagurka (2004) modelled the bounce as a mass-spring-damper system, giving the time of contact for each bounce. It provides a little more realism by capturing some aspects of the bounce sound, Stoelinga (2007) did a detailed analysis of bouncing and rolling sounds. It has a wealth of useful information, and deep insights into both the physics and perception of bouncing, but stops short of describing how to synthesize a bounce.

To really capture the sound of a bounce, something like modal synthesis should be used. That is, one should identify the modes that are excited for impact of a given ball on a given surface, and their decay rates. Farnell measured these modes for some materials, and used those values to synthesize bounces in Designing Sound . But perhaps the most detailed analysis and generation of such sounds, at least as far as I’m aware, is in the work of Davide Rocchesso and his colleagues, leaders in the field of sound synthesis and sound design. They have produced a wealth of useful work in the area, but an excellent starting point is The Sounding Object.

Are you aware of any other interesting research about the sound of bouncing? Let us know.

Next week, I’ll continue talking about bouncing sounds with discussion of ‘the audiovisual bounce-inducing effect.’

References

  • Aguiar CE, Laudares F. Listening to the coefficient of restitution and the gravitational acceleration of a bouncing ball. American Journal of Physics. 2003 May;71(5):499-501.
  • Farkas N, Ramsier RD. Measurement of coefficient of restitution made easy. Physics education. 2006 Jan;41(1):73.
  • Luck, J.M. and Mehta, A., 1993. Bouncing ball with a finite restitution: chattering, locking, and chaos. Physical Review E, 48(5), p.3988.
  • Nagurka, M., Shuguang H,. “A mass-spring-damper model of a bouncing ball.” American Control Conference, 2004. Vol. 1. IEEE, 2004.
  • Schwarz O, Vogt P, Kuhn J. Acoustic measurements of bouncing balls and the determination of gravitational acceleration. The Physics Teacher. 2013 May;51(5):312-3.
  • Stoelinga C, Chaigne A. Time-domain modeling and simulation of rolling objects. Acta Acustica united with Acustica. 2007 Mar 1;93(2):290-304.

Sound Talking – 3 November at the London Science Museum

On Friday 3 November 2017, Dr Brecht De Man (one of the audio engineering group researchers) and Dr Melissa Dickson are chairing an unusual and wildly interdisciplinary day of talks, tied together by the theme ‘language describing sound, and sound emulating language’.

Despite being part of the Electronic Engineering and Computer Science department, we think about and work around language quite a lot. After all, audio engineering is mostly related to transferring and manipulating (musical, informative, excessive, annoying) sound and therefore we need to understand how it is experienced and described. This is especially evident from projects such as the SAFE plugins, where we collect terms which describe a particular musical signal manipulation, to then determine their connection with the chosen process parameters and measured signal properties. So the relationship between sound and language is actually central to Brecht’s research, as well as of others here.

The aim of this event is to bring together a wide range of high-profile researchers who work on this intersection, from maximally different perspectives. They study the terminology used to discuss sound, the invention of words that capture sonic experience, and the use and manipulation of sound to emulate linguistic descriptions. Talks will address singing voice research, using sound in accessible film for hearing impaired viewers, new music production tools, auditory neuroscience, sounds in literature, the language of artistic direction, and the sounds of the insane asylum. ‘Sounds’ like a fascinating day at the Science Museum!

Register now (the modest fee just covers lunch, breaks, and wine reception) and get to see

  • Maria Chait (head of UCL Auditory Cognitive Neuroscience lab)
  • Jonathan Andrews (on soundscape of the insane asylum)
  • Melissa Dickson (historian of 19th century literature)
  • Mariana Lopez (making film more accessible through sound)
  • David Howard (the singing voice)
  • Brecht De Man (from our group, on understanding the vocabulary of mixing)
  • Mandy Parnell (award winning mastering engineer)
  • Trevor Cox (categorising quotidian sounds)

In addition, there will be a display of cool sound making objects, with a chance to make your own wax cylinder recording, and more!

The full programme including abstracts and biographies can be found on www.semanticaudio.co.uk/events/soundtalking/.