IEEE Computational Intelligence Magazine - November 2021 - 72

of training. Therefore, it is unnecessary to
store and transmit the decoding networks
secretly. Additional data can be extracted
correctly with the correct embedding
key. The novelty and contributions of this
paper are summarized as follows:
1) This paper focuses on a new form
of data hiding by embedding additional
data into CapsNets without
visibly decreasing its detection accuracy.
This is achieved by embedding
additional data during the training
process instead of modifying the
network parameters after training.
2) This paper proposes a customized
data hiding scheme for CapsNets,
which is able to transmit different
additional data with multiple receivers.
A receiver can obtain the data for
himself using the correct embedding
key but has no knowledge of other
parts of the additional data.
3) The practicability of the proposed
scheme is satisfactory. This is because
the parameters of data decoding networks
are determined by embedding
keys instead of training. In this way, it
is unnecessary to store and transmit
the decoding networks secretly.
II. Related Work
In this section, some related work is
introduced, including the data hiding
technique and CapsNets employed for
data hiding in this paper.
A. Data Hiding
Data hiding is a technique to imperceptibly
embed additional data into digital
media, which can be mainly categorized
into watermarking and steganography
according to different applications.
Watermarking is the process of marking
digital media for copyright protection
[20], while steganography is mainly
developed for covert communication
[21]. There are many practical applications
of data hiding, e.g., biometric
watermarking, chip and hardware protection,
combination of watermarking and
encryption, and cloud computation [22].
Watermarking can be used in many
applications, such as digital forensics,
integrity authentication, copyright protection,
and source tracking. Popular
media employed for watermarking
include audio, images, and video, to
name a few. To embed watermarked
data, the redundancy components of the
cover media are often exploited, including
but are not limited to the correlation
of pixels in the spatial domain [23]
and the frequency spectrum characteristic
in the transform domain [24]. Watermarking
in neural networks is becoming
a popular topic in information security
since it is a straightforward way to protect
the intellectual property of neural
networks using digital watermarking
[14]. Many watermarking schemes are
designed for neural networks [15] by
embedding watermarking in weight
parameters in networks or using the
backdooring technique.
Modern steganography hopes to
minimize the embedding distortion on a
cover image with a given distortion
function [25]. Many distortion functions
for steganography have been reported
[26]. Recently, generative steganography
has been developed in which stego
images are generated directly according
to additional data [27]. Some researchers
generate stego images with the help of
GANs or encoder-decoder networks
[28]. In [29], additional data were transformed
directly into a fingerprint image
based on a piece of hologram phase
constructed by the additional data. The
authors in [30] focused on linguistic
steganography. They captured the correlation
between words using a RNN and
generated stego text directly from the
additional data.
This paper focuses on data hiding in
a neural network, which is able to transmit
different additional data with multiple
receivers. Details are described in
Section III.
B. Capsule Network
The fundamental architecture of CapsNets
[8] designed for the MNIST dataset
is shown in Fig. 3, which is equipped
with only two convolutional layers
(Conv1 and PrimaryCaps) and one fully
connected layer (DigitCaps). In the
Conv1 layer, there are 256 convolutional
kernels sized 99# with a stride of 1 and
ReLU activation. This layer extracts local
pixel features that are used as inputs to
the PrimaryCaps layer. PrimaryCaps is a
convolutional capsule layer that contains
32 capsules. Each capsule contains 8 convolutional
kernels sized 99# with a
stride of 2 and ReLU activation. Finally,
the DigitCaps layer contains 10 capsules
sized 16D. Each capsule represents a digit
class connected with all the capsules in
PrimaryCaps, and the capsule norm
DightCaps
ReLU Conv1
ReLU
Conv1
9 × 99×9
9 × 99×9
20
6
FIGURE 3 Architecture of CapsNets for MNIST.
256
Primary
Caps
Primary
Caps
8
32
16
10
||L2||
10
...
72 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | NOVEMBER 2021
IEEE Computational Intelligence Magazine - November 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - November 2021
