Signal Processing - May 2017 - 39

dominance, whereas the effect that a single auditory event is
perceived when there are two sound events is called fusion. The
effect that the discrimination of the direction of the lagging sound
source is suppressed is termed lag discrimination suppression.
These three are collectively known as the precedence effect [16].
Another important binaural cue is interaural coherence (IC),
which is a measure of the coherence of signals received by the
two ears. IC is high for sounds coming directly from the source,
where the two ear signals are highly correlated, and low in the
diffuse sound field, where the correlation is low. Therefore, IC
provides information about the level of reverberation and thus
about the spaciousness of the environment.

The history of perceptually motivated
spatial audio
Binaural audio and multichannel stereophony are two of the
most common spatial audio technologies, predating more
recent technologies such as WFS by more than half a century.
Binaural audio has found extensive use and received renewed
interest, especially for virtual reality (VR) applications, while
multichannel systems have been the de facto standard for
home entertainment and automotive audio systems.
Simultaneously, due to the popularity and market dominance
of two-channel audio formats, stereophony using two loudspeakers is still commonly employed.

Binaural audio
Binaural audio is based on a simple assumption: if the signals
that would be received at the ears of a listener as a result of an
acoustic event are provided to the listener with sufficient accuracy, the person will perceive an auditory event corresponding
to the original acoustic event. These ear signals can either be
recorded with microphones implanted in the ear canals of an
artificial human head, such as Knowles Electronic Manikin
for Acoustic Research or Neumann KU-100, or synthesized
using signal processing methods. In both cases, the signals are
usually presented over a pair of headphones.
The microphones used for recording binaural audio are
also termed dummy head microphones and are manufactured
to resemble a typical human head. The external ears of these
microphones are typically molded in silicon and are modeled
after the external ears of humans who have exceptional spatial
hearing acuity. The recorded signals need to be played back
using headphones equalized appropriately with free-field or
diffuse-field equalization, depending on the environment in
which the recording was made [1].
Binaural synthesis is based on the knowledge of the acoustic
transfer paths between the source and the two ears. These paths
are characterized by their impulse responses, referred to as the
head-related impulse response (HRIR) and head-related transfer
function in the frequency domain. For each source position, there
will be two of them, one for the left ear and one for the right.
When HRIRs are convolved with dry source signals, the resulting
signals will incorporate the necessary binaural cues for the given
source position. In the case of a sound field created by P sources
in the far field, the right and left ear signals can be synthesized as

x L (n) =

/ x p (n) * h L,

i p, z p

(n),

(1)

i p, z p

(n),

(2)

p =1

x R (n) =

/ x p (n) * h R,

p =1

where x p (n) is the pressure signal due to source p; h L, i p, z p (n)
and h R, i p, z p (n) represent the HRIRs for the left and the right
ears for a source at a direction (i, z), where i and z are the
azimuth and elevation angles, respectively; and * denotes convolution. This approach assumes that the acoustical system
consisting of these sources and the listener is linear and time
invariant and that the resulting left and right ear signals
provide the necessary spatial hearing cues pertaining to the
acoustic field that would be generated by these P sources.
In free field, HRIRs can be considered finite and are typically up to 12-ms long, corresponding to approximately 512
samples at the 44.1-kHz sampling rate. This does not present
a significant computational cost for a single component. However, as the number of components increases, such as when a
source and its reflections in a room are being rendered, the
computational cost of convolution becomes an important
bottleneck. To overcome this limitation, different filter design
approaches have been proposed, e.g., [17]. These filters are
designed to capture salient binaural cues while significantly
reducing the computational cost.
Two essential requirements of binaural synthesis are 1) the
availability of a set of HRIR measurements densely sampled on a
spherical shell and 2) the match between these HRIRs and the actual
HRIRs of the listener. Regarding the first requirement, interpolation methods such as kernel regression [18] can be used to increase
the granularity of the available directions. The second requirement
necessitates the measurement of individualized HRIRs, which is
both time consuming and costly. For that reason, many existing
research-grade and commercial solutions use generic HRIRs. This,
however, is not an ideal solution, since there are significant differences
between the spectra of the generic HRIRs and individual HRIRs of
the listener, and these cues are essential for elevation perception [13].
Practical setups that allow quick measurement of HRIRs around a
geodesic sphere surrounding the listener's head have recently been
developed [19]. There also exist commercial products that allow
tailoring a stored set of HRIRs based on head size (https://www
.ossic.com). However, head size alone can improve only the ITD and
ILD cues provided by the system, not the spectral cues used in the
perception of source elevation.
Binaural synthesis also allows interactivity if the position
and orientation of the listener's head can be tracked [20]. Highprecision and high-accuracy magnetic trackers had been the
de facto method for tracking a listener's head. Recent developments made it possible to track a user's head with inexpensive
devices (http://www.3dsoundlabs.com). These developments
make binaural synthesis an excellent solution for VR applications. For binaural synthesis, a side effect of system errors-
such as a pair of improperly equalized headphones, an HRIR
set that does not well match the HRIRs of the user, or inaccurate head tracking-is inside-the-head localization [21]. This

IEEE Signal Processing Magazine

May 2017

https://www http://www.ossic.com http://www.3dsoundlabs.com

Table of Contents for the Digital Edition of Signal Processing - May 2017