IEEE Computational Intelligence Magazine - May 2020 - 50

are numerous. In some domains, espe-
cially medicine and finance, each insti-
tute only has access to limited amount
of data, and large datasets are often
crowdsourced. Even though these datas-
ets for machine learning tasks enable
faster commercial or scientific progress,
the critical and sensible demand for
preserving individual privacy from inva-
sion continues to rise in the crowd,
companies and the government.
Ideally, the sensitive individual infor-
mation should not be leaked in the pro-
cess of training machine learning mod-
els. In other words, we allow the
parameters of machine learning models
to learn general patterns (people who
smoke are more likely to suffer from
lung cancer), rather than facts about spe-
cific training samples (he had lung can-
cer). Unfortunately, shallow models like
support vector machine and logistic
regression are capable of memorizing
secret information of the training data-
set [2]. Deep models like convolutional
neural networks are able to exactly
memorize arbitrary labels of the training
data [3]. Recent attacks against machine
learning models as in [2], [4]-[8]
emphasize the implicit risks and catalyze
an urgent demand for privacy preserv-

M(D)
M(D ′)

Differential Privacy Mechanism

Instance

Inference

Ratio Bounded
by e P

ing. For examples, Shokri et al. [4]
designed a membership inference attack
that can estimate whether the training
dataset contains a specific data record via
the black-box access to the model.
Fredrikson et al. [5] presented a model
inversion attack that can reveal individ-
ual faces given the API of the face rec-
ognition system and the name of the
user to be identified. In [6], the decision
probability of the classification model
can be used for the model extraction
attack, which implicitly steals the sensi-
tive training data. Attackers abuse the
pharmacogenetic model to inversely
infer the patients' genetic markers [7].
Adversaries can maliciously acquire
unexpected but useful information from
the machine learning classifiers [8].
To alleviate the possible privacy threats
to data owners, an attractive and viable
method is to involve the privacy-preserv-
ing techniques into machine learning
approaches. In early works, it is prevalent
to anonymize the data before analyzing
data including k-anonymity [9], l-diversity
[10], t-closeness [11], which remove pri-
vate details or replace them with random
values. Nevertheless, they are not always
sufficient and only provide privacy guar-
antee to a certain extent, especially when

Dataset
Algorithms
Learning

Released Models
0 1 2 ...8 9
Prediction

FIGURE 1 A graphical illustration of incorporating differential privacy into machine learning for
privacy preserving.

IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | MAY 2020

adversaries own auxiliary individual infor-
mation in the sensitive dataset. Besides,
anonymizing is not applicable to high-
dimensional or diverse input datasets due
to its strong theoretical and empirical lim-
itations [12], [13]. As a solid privacy
model, differential privacy [14] has
recently been considered as a promising
strategy for privacy preserving in machine
learning. There are roughly three major
reasons: (1) Differential privacy can pro-
vide a provable privacy guarantee for
individuals, which benefits from the most
solid theoretical basis compared with
other privacy-preserving models [9]-[11],
[15], [16]. (2) Differential privacy achieves
privacy preserving in machine learning by
adding a calibrated amount of noise to
the model or output results according to
the concrete mechanisms instead of sim-
ply anonymizing the individual data. (3)
Differential privacy can make a graceful
compromise between privacy and utility
by adjusting a privacy budget index, in
which the smaller the value of the privacy
budget, the stronger privacy guarantee it
provides. For data owners, differentially
private machine learning further ensures
that the adversaries are incapable to infer
any information about a single record
with high confidence from the released
machine learning models or output
results, even if an adversary knows all the
remaining records in this dataset. A graph-
ical illustration of incorporating differen-
tial privacy into machine learning for pri-
vacy preserving is shown in Figure 1.
In the past decade, we have witnessed
the rapid advances of new methods about
differentially private machine learning.
This is entirely due to the remarkable
capability of differential privacy in pro-
viding effective and efficient approaches
for solving the problem of privacy pre-
serving, by utilizing the basic mechanisms
such as Laplace mechanism [14], expo-
nential mechanism [17], and functional
perturbation mechanism [18]. These dif-
ferential privacy mechanisms can com-
bine the strength of differential privacy to
satisfy the requirement of privacy pre-
serving for non-private prototypes of a
wide range of machine learning tech-
niques. There are only a few existing
reviews on the topic of differentially

IEEE Computational Intelligence Magazine - May 2020

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2020