IEEE Computational Intelligence Magazine - August 2022 - 15

on machine and deep learning and operate on
the cloud or in mobile apps [3]. Unfortunately,
the processing of data acquired by users, companies,
or stakeholders by third-party software services
may severely impact privacy when
sensitive data are involved (e.g., medical diagnoses,
political or personal opinions, and confidential information)
[4]. The need to combine privacy with intelligent services sheds
light on one of the most relevant scientific and technological
challenges of the coming years: How can software services and mobile
apps that provide intelligent functionalities (through machine and deep
learning solutions) be designed while guaranteeing the privacy of user data?
This is a crucial question that research has begun to address from
several perspectives, including scientific [5], technological [6], [7],
and legislative [8]. Table I presents a comparison of the main
approaches provided in the literature for integrating privacy constraints
with intelligent processing abilities.
Interestingly, among these families of solutions, homomorphic
encryption (HE) is the only one that guarantees both the ability
to process encrypted data as well as to operate without requiring
multiple rounds of client-server computation/communication.
HE schemes represent a special type of encryption that
allows (a set of) operations to be performed on encrypted data.
Specifically [9], an encryption function E and its decryption
function D are homomorphic w.r.t. a class of functions F if,
for any function
f F ,! one can construct a function g such
that () (( ( ))) for a set of input m.
fm D gEm=
Due to HE's ability to perform operations on encrypted
data without multiple rounds of client-server communications,
it is particularly suitable for consideration in the " as-a-service "
computing paradigm, which requires high standards of privacy
and data confidentiality. Indeed, integrating HE with machine
and deep learning solutions could lead, for example, to the
design of a cloud-based diagnosis system that is able to process
X-ray images previously encrypted by a patient. The encrypted
results (e.g., an index measuring the presence of potentially
critical health threats) would be sent back to the patient, who
would be the only one able to decrypt them.
Unfortunately, this ability comes at the expense of three
drawbacks: First, only a subset of operations (mainly addition and
multiplication) is allowed in most of HE-based processing systems;
second, the length of the processing pipeline (i.e., the
amount and type of operations to be executed) is restricted; and
third, the memory and computational demand of HE-based processing
systems are much higher than those of traditional systems.
These three drawbacks are particularly relevant in a scenario
where deep learning solutions are considered, since deep learning
models are typically characterized by a long pipeline of processing
layers that comprises various types of nonlinear
operations. For this reason, deep learning models and solutions
must be completely redesigned and redeveloped to consider the
constraints of HE schemes. Only a few studies have proposed
addressing this issue with effective solutions in highly specific
fields [10], and a general approach to HE for deep learning is
still missing. Therefore, the aim of this study was twofold:
Homomorphic encryption schemes represent a special
type of encryption that allows (a set of) operations to
be performed on encrypted data.
❏ To introduce HE for machine and deep learning by
describing HE encoding/encryption mechanisms and the
operations on plaintexts/ciphertexts;
❏ To provide a methodology for the step-by-step design of
privacy-preserving convolutional neural networks (CNNs)
based on HE. The goal of this methodology is to trace the
path in the design of HE-based machine and deep learning
solutions for supporting privacy-preserving intelligent processing
in cloud-based services or mobile apps.
To achieve these aims, this study complemented theory with
examples and code by applying the proposed methodology to
the design of a privacy-preserving version of the well-known
LeNet-1 CNN [11]. Experimental results are presented regarding
the effectiveness and efficiency of the privacy-preserving
LeNet-1 on two benchmark datasets for image classification.
Furthermore, the research challenges and the software resources
available for the design of privacy-preserving deep learning
solutions are detailed and commented on. In addition, all of the
codes used in this study have been made available to the scientific
community as a public repository.1
The remainder of this paper is organized as follows. Section
II introduces the Brakerski-Fan-Vercauteren (BFV) scheme for
HE together with the encoding/decoding mechanisms and privacy-preserving
operations. Section III introduces the proposed
methodology for the design of privacy-preserving CNNs with
HE, and then Section IV details the application of this methodology
to the well-known LeNet-1 CNN. Finally, the research
challenges and software resources available for privacy-preserving
deep learning are presented in Sections V and VI, respectively,
before the conclusions of the study are drawn in Section VII.
II. Homomorphic Encryption: The BFV Scheme
This section illustrates the main characteristics of HE schemes
and provides concrete examples of its algebraic peculiarities.
An HE scheme is an encryption scheme that supports the
1https://github.com/AlexMV12/Introduction-to-BFV-HE-ML
TABLE I Comparison of methodologies for privacy-preserving
machine learning.
Ability to
Process
Encrypted
Data
Homomorphic encryption
Multi-party computation
Group-based anonymity
Differential privacy
Yes
Yes
No
No
Processing Without
the Need for
Multiple Rounds of
Communication
Yes
No
Yes
Yes
AUGUST 2022 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 15
https://github.com/AlexMV12/Introduction-to-BFV-HE-ML
IEEE Computational Intelligence Magazine - August 2022

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - August 2022
