IEEE Computational Intelligence Magazine - November 2021 - 75

procedures of data embedding, the proposed
scheme is summarized in Algorithm 1.
The parameters of the FC layers in
the decoding networks are obtained
using embedding keys instead of training.
Before training, the parameters of
decoding networks are produced using
embedding keys. These parameter values
are kept unchanged during the process
of training; consequently, it is unnecessary
to store the decoders secretly. Additional
data can be extracted correctly
with correct embedding keys, and the
practicability is thus satisfactory. In
addition, additional data are embedded
during the process of training instead of
modifying the network parameters after
training. For this reason, the detection
accuracy of the neural network will not
be decreased visibly, and security performance
can be guaranteed (the existence
of additional data cannot be exposed). To
verify this, the parameter distributions of
the original and stego CapsNets are
compared on the MNIST dataset, as
shown in Fig. 6, where
b 1 ,=
t 10000=
n 1= ,
, and the batch size is 50. In
data hiding, as shown in Equation (11),
the difference between an original and
stego distribution can be measured using
the KL divergence [33], which is a statistical
test from information theory measuring
the distance of two distributions.
Dp xp xp xxx=
x
s
pxx
pxs
(( )( )) / ()log
px
px
x
s
()
()
(11)
where () and () are the distributions
of the cover and stego objects,
respectively. The value of KL divergence
is always nonnegative and is 0 if and
only if the two distributions are equal. A
smaller value of KL divergence means
higher similarity between the two distributions.
The KL divergence values
between the distributions in Fig. 6 are
listed in Table II. The results indicate
that the distributions of the original and
stego parameters are similar since the
KL divergence values are small. In this
case, it is difficult to judge whether a
given network contains additional data
or not. The existence of additional data
therefore cannot be exposed so as to
guarantee the security of the proposed
A. Parameter Determination
The parameter b in Equation (7),
which is the weight of the loss function,
The embedding capacity of the proposed scheme is
6000 bits, which is satisfactory for data hiding.
scheme. It can also be noticed that the
differences between the original and
stego distributions of sj
and vj
are larger
than those of the other elements. The
reason may be that the redundancy in
coupling coefficients cij
shown in Equations (1) and (2), sj
vj are determined by c .ij
and vj
is trivial. As
and
In this case,
the differences between the original and
stego distributions of sj
large if the redundancy in cij
will be
is trivial. In
future work, a scheme to minimize the
distribution differences of all elements
can be developed.
IV. Experimental Results
To verify the effectiveness of the proposed
scheme, some experiments are
conducted in this section. To the best of
the authors' knowledge, the data hiding
method for multiple receivers using
neural networks has not been reported
in the literature. Thus, this section mainly
examines the performance of the proposed
scheme.
All the experiments are implemented
by TensorFlow and trained under the
environment of Python 3.8 on a Windows
10 system with an NVIDIA
GeForce RTX 2080 Ti GPU with
11GB of memory. The Adam optimizer
[34] is used for optimization.
is important since it balances the detection
accuracy of CapsNets and the performance
of the data hiding scheme. To
determine the value of
b , a group of
experiments for the case of one receiver
()n 1= are conducted on the MNIST
dataset with a capacity of 10000 bits
(),
t 10000=
2 routing iterations, and a
batch size of 50. The extraction error of
data and detection accuracies of CapsNets
containing additional data are
shown in Fig. 7.
There is a trade-off between the
extraction error of data extraction and
the detection accuracy of CapsNets. A
large value of b is advantageous to data
extraction but disadvantageous to detection
accuracy, and vice versa. Fig. 7(b)
shows that the detection accuracy clearly
decreased when
b 2 15 Meanwhile,
..
it can be seen from Fig. 7(a) that the
extraction error also performs well at
..
b 15=
With overall consideration, the
value of b is determined to be 1.5 to
guarantee satisfactory detection accuracy
and extraction error simultaneously.
B. Embedding Capacity
For data hiding, embedding capacity is
one of the most important indicators. In
this subsection, the embedding capacity
of the proposed scheme is tested. For
the case of one receiver (),
n 1= the
length t of additional data M is set as
{, ,, },
500 1000 7500f
Algorithm 1 Procedures of data embedding.
Input: Additional data {, ,, };MM Mn12
f
and the number
of routing iterations is set as {2, 3, 4}.
Embedding keys: {, ,, };KK Kn12
f
Architecture
of CapsNets; Images for training and corresponding labels.
Output: Trained CapsNets with different additional data for n receivers.
1) Construct CapsNets architecture for data hiding as described in Subsection III-B;
2) Produce the parameters of decoding networks (the n FC layers shown in
Fig. 5) using {, ,, },KK Kn12
f
respectively. These parameter values are kept
unchanged during the process of training;
3) With the guidance of {, ,, },MM Mn12
f
train new CapsNets using the given
images and labels to minimize the total loss L calculated in Equation (7);
4) The obtained network is the desired output: " Trained CapsNets with different
additional data for n receivers. "
NOVEMBER 2021 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 75
IEEE Computational Intelligence Magazine - November 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - November 2021
