IEEE Circuits and Systems Magazine - Q4 2019 - 31

produce expressive music. An automatic music generation method was proposed based on the combination of the symbolic models and the raw audio models.
The musical samples achieved7 gave a high level of the
naturalness and continuity. In another work, a powerful WaveNet-style autoencoder model was proposed to
condition an autoregressive decoder for capturing longer term structure without external conditioning [101].
The model can learn a manifold of embeddings, which
allows for morphing between instruments and interpolating in timbre. In [102], autoregressive discrete autoencoders (ADAs) were explored to train autoregressive
models to learn long-range correlations in raw audio
waveforms. This technique can be employed to unconditionally produce piano music directly in the raw audio domain. In [103], a novel singing synthesizer was
proposed based on a modified version of the WaveNet
network. Compared to other works, the features produced by a parametric vocoder were modeled in this
work. Given a musical score with lyrics, this approach
can generate a synthetic singing voice that can learn
both timbre and expression.
IV. Hybrid Model: Generative
Adversarial Networks
Generative adversarial networks (GANs) are a class of
neural networks developed for unsupervised machine
learning. Generally, a standard GAN consists of two
main components: a generator G and a discriminator
D, which are trained by contesting with each other under a zero-sum game strategy [43]. The discriminator is
trained by traditional supervised learning algorithms to
perform two-class classification. In contrast, the generator is trained to deceive the discriminator by creating
samples of the same distribution as the training data
[52], [104].
These types of generative models have been employed
in many applications, including high-quality image generation [105], [106], speech/image synthesis [107]-[109], im7

age translation [110], [111], semantic segmentation [112],
[113] and object detection [114], [115].
A. How GANs Work
The final goal of a GAN is to learn how to generate new
samples from the training samples. Assume that the probability distribution of the training data is p^ x h . In traditional generative models, we sample from p^ x h to obtain
new samples whereas GANs attempt to learn a mapping
from a random input (or noise) to the training sample.
Take image generation as an example, as shown by Fig. 14
the generator is denoted as G^ z; i g h where i is the network parameter and p z ^ z h is a prior on input noise variables. The input to the generator G is a 1-dimension random vector, z, and the output is an image produced by
G. To force images that are produced to be in the same
distribution as the training images, a discriminator is connected to G and denoted as D^ x; i d h . The input of D is
selected randomly from either a real image or a generated
y

H3

Hidden Layer
RBM_3

V3/H2

Hidden Layer
RBM_2

V2/H1

Hidden Layer
RBM_1
Input Layer

V1

x
Figure 13. The network architecture of a standard DBN.

http://people.bu.edu/bkulis/projects/music/index.html

Real Samples
pdata
Discriminator
D (x;θd )
Noise z

Generator
G (z;θg )

Real/Fake?

Fake Samples
pg

Figure 14. Block diagram of a GAN.

FOURTH QUARTER 2019

IEEE CIRCUITS AND SYSTEMS MAGAZINE

31
IEEE Circuits and Systems Magazine - Q4 2019

Table of Contents for the Digital Edition of IEEE Circuits and Systems Magazine - Q4 2019
