IEEE Signal Processing - March 2018 - 12

special reports
John Edwards

Signal Processing Supports a New Wave of Audio Research
Spatial and immersive audio mimics real-world sound environments

n an era of ubiquitous video, audio is
often relegated to a secondary role. Yet
electronic audio in all of its various
forms is now staging a strong comeback
as listeners, increasingly dissatisfied
with the output of highly compressed
audio files and streams, seek higher
sound quality on all types of fixed and
mobile platforms.
Signal processing is largely responsible for taking audio in all of its forms
to new levels of quality. On the cutting
edge of audio research is spatial audio,
or ambisonics, a technique that takes a
dimensional approach to sound, mimicking the way people hear in real life. By
channeling the characteristics of sound as
it travels through space and time, ambisonics envelops listeners inside a threedimensional (3-D) audio sphere that
makes recorded music sound startlingly
natural in whatever setting it's used.

Orchestra Project" researchers were able
to present what amounted to a virtual
recreation of the live performance.
Researchers reproduced a performance
of the 19th century opera La Vierge (The
Virgin) using computerized acoustical data,
enhancing the experience with computergenerated virtual navigation. The 3-D visualizations were created with immersive
architectural renderings that guide the
viewer through the complex acoustics of
the medieval Gothic cathedral (Figure 1).
"Our work employs geometrical
acoustic methods for the room acoustic impulse response synthesis, based
on ray-tracing methods," states project
leader Brian F.G. Katz, research director
at the Institute Jean Le Rond d'Alembert,
Pierre and Marie Curie University. The
technology's 3-D room impulse respons-

es are generated in high-order ambisonics, second or third order, depending on
central processing unit (CPU) limitations. "These responses are then decoded
and convolved with head-related transfer
function (HRTF), which represents the
acoustic scattering properties of the human torso, head, and outer ears," Katz
explains. The processing technique, binaural rendering, permits a 3-D acoustic
rendering of a sound source within a specific architectural space.
"Our work in room acoustic simulations for virtual reality, and improving
binaural rendering through individualization techniques, are both long-term
research projects," Katz says. He notes
that several years of experimentation
and results have shown considerable
advances in the perceptual realism of

Virtual performances
in the cathedral

Digital Object Identifier 10.1109/MSP.2017.2784881
Date of publication: 7 March 2018

Ghost orchestra Project/LIMsI

Visitors to the Cathedral of Notre Dame
in Paris in early 2017 had a good chance
of hearing what sounded like live music
being played, even when there were no
musicians on site. What they were actually hearing was a recording of a live
concert held several years earlier at the
cathedral. With the assistance of several
computer models based on the live performance, along with highly detailed
room acoustic simulations, "Ghost
Figure 1. A screenshot of a visual rendering model of the Notre Dame Cathedral, based on plans,
laser scan data, and site visits.

IEEE Signal Processing Magazine

March 2018

Table of Contents for the Digital Edition of IEEE Signal Processing - March 2018