IEEE Circuits and Systems Magazine - Q4 2019 - 24

samples were generated.3 In [51], an interactive musical audio synthesis system4 based on ANNs was introduced. In this system, frames of low-level features were
learned by an ANN and a high level representation of
the musical audio was learned through an autoencoder.
The purpose of this paper is to offer a timely overview of the applications of deep learning approaches
to audio generation problems, especially speech generation. Deep learning algorithms involved are categorized as either discriminative or generative methods,
taking GANs as a distinctive hybrid method. The background on the different deep learning architectures
and up-to-date applications are provided to readers in
this area. The rest of the paper is organized as follows.
In Section II we start by reviewing recent deep discriminative algorithms and the remarkable progress
3
4

http://soundcloud.com/ann_synth
https://github.com/woodshop/deepAutoController

Input

Hidden

Output

x1
y1
x2
y2
x3
y3
b

Figure 4. A single (one hidden) layer MLP.

wx + b

Figure 5. Computation procedure at each node.
24

II. Discriminative Algorithms
The distinction between discriminative and generative models is the probability distribution modeled.
Generally, an output variable y needs to be estimated
by a standard pattern recognition model given an input variable x. A deep discriminative algorithm, like a
DNN, utilizes multi-layer hierarchical architectures to
directly compute the probability of y given an x, i.e.
to estimate p ^y|x h . The most common discriminative
models used in machine learning include logistic/linear regression, support vector machines (SVMs), random forests and neural networks. With different structural elements, deep discriminative models contain
various implementations in terms of tasks and functions. In this paper we only focus on the discriminative
models with deep hierarchical architectures such as
the MLP, CNNs and RNNs.
A. Deep Discriminative Architectures
1) Multi-Layer Perceptron
A multi-layer perceptron, as a type of feedforward artificial neural network, consists of at least three layers of
computing units (also known as "nodes" or "neurons").
A diagram of a single layer MLP is shown in Fig. 4. Except for an input and an output layer, one or more hidden layers can be inserted as required. Each node in a
hidden layer or output layer contains a nonlinear activation function depending on how the network has been
configured. The computation process of the lth layer is
shown in Fig. 5 and is described by:
y l = f l ^w l x l + b l h

w1
x2

in speech synthesis and acoustic signals generation.
While the same understanding and analysis are shown
for deep generative algorithms in Section III. Specific
discussion of GANs and their variations are reported in
Section IV. The reasons why deep learning can be beneficial for pattern recognition problems and issues to
be studied further are given in Section V. We conclude
the paper in Section VI.

IEEE CIRCUITS AND SYSTEMS MAGAZINE

Output

(1)

where x are inputs to the network while weights and biases are denoted as w and b, respectively. The activation function f is nonlinear to enhance the capacity of
the modeling representation, especially beneficial for
nonlinear classification problems. Like in [26], the backpropagation algorithm is employed to update weights
and biases of all the layers with the loss calculated.
Specific objective functions are set to estimate the distance between the prediction and the actual value. With
trained network parameters, we can use the MLP to
FOURTH QUARTER 2019

IEEE Circuits and Systems Magazine - Q4 2019

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q4 2019