IEEE Circuits and Systems Magazine - Q4 2019 - 28

the musical score and its acoustic features was modeled
in frames by a DNN. Subjective experimental results
demonstrated that the DNN-based singing voice synthesizer outperformed the conventional HMM-based
system in terms of naturalness. In listening tests, the
deep learning based singing synthesizer can generate a
sound quality on-par or exceeding the previous stateof-the-art concatenative methods. The timbre and fundamental frequency carried in the natural songs can be
jointly modeled by deep learning approaches, which allows for much faster experimentation because of the superior tools like GPUs [78]. Another interesting application is the conditional rhythm composition [79], in this
work a deep network was proposed as a combination of
LSTMs and feed forward layers. This architecture was
applied to learn long drum sequences from the metrical
information and bass lines.
III. Generative Algorithms
For generative algorithms, they provide the joint probability distribution of the input and the output of the
deep learning model. In other words, generative models aim to estimate p ^x, y h . However, p ^y|x h can also be
obtained with Bayes' theorem to indirectly perform pattern classification tasks [42]. Several widely used generative algorithms are Gaussian Mixture Model (GMM),
Hidden Markov Model (HMM), Restricted Boltzmann
Machine (RBM), Deep belief Network (DBN) and Variational autoencoders (VAEs).
A. Deep Generative Architectures
1) Variational Autoencoder
The variational autoencoder (VAE) [40] is derived from
the original autoencoder but with stronger assumptions concerning the distribution of latent variables. In

Encoder

order to understand the VAEs we need to first introduce the autoencoder.
An autoencoder is a type of NN that learns data codings in an unsupervised manner [80]. Dimensionality reduction is the main application of autoencoders due to
the capability of representing a set of data [36]. Fig. 10 is
a classical schematic structure of an autoencoder with
three fully-connected hidden layers.6 This is identical to
the MLP which is built by many single layer perceptrons
as a feedforward neural network. Note that for autoencoders, the number of nodes on the output layer is the
same as the input layer. Generally, the size of the hidden layers are smaller than that of the input or output
layers. This unique type of architecture is designed for
compressing inputs to a compact code and uncompressing that code into the outputs. The outputs are expected
to be closely matched to the inputs. The two main components of an autoencoder for completing the compress
and the uncompress operations are called the encoder
and the decoder, respectively. The output layer of the
encoder (that is also the input layer of the decoder) is
usually referred to as a code, latent variables or a latent
representation. In pattern recognition tasks like image
processing, the latent representation can be used as extracted features for a further supervised learning. While
in unsupervised learning tasks, variations of the autoencoder (such as VAEs) can be used to generate new data
from the training inputs. Therefore, autoencoders have
emerged as one of the most popular approaches in the
field of machine learning in recent years [81]-[85].
Although standard autoencoders have been successfully applied, there are several problems limiting the development of autoencoders. The most fundamental one
6

The cat image used in Fig. 10 is from the YouTube video: https://www
.youtube.com/watch?v=YCaGYUIfdy4

Latent
Variables

Decoder

Figure 10. A standard autoencoder with three fully-connected hidden layers.
28

IEEE CIRCUITS AND SYSTEMS MAGAZINE

FOURTH QUARTER 2019

https://www.youtube.com/watch?v=YCaGYUIfdy4 https://www.youtube.com/watch?v=YCaGYUIfdy4

IEEE Circuits and Systems Magazine - Q4 2019

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q4 2019