IEEE Computational Intelligence Magazine - November 2021 - 80

meets (, )N 0 1
2
noise x2 meets (, ).U 0 2z
z and that of uniform
The two kinds
of noise are obtained using the functions
" np.random.normal() " and " np.random.
rand() " in Python.
For different values of 1z and
z2 , the
results of extraction error e on the
MNIST dataset with 2 routing iterations,
batch size 50, n 1= and
b 15 .=
are shown in Fig. 11. Intuitively, correct
extraction of additional data can be
guaranteed as long as noise is not powerful.
This is verified in Fig. 11. In particular,
additional data can be extracted
correctly when
z1 00 . 81 for Gaussian
noise and z2 02 . 51 for uniform noise.
This means that the proposed scheme is
robust to the Gaussian noise that is not
stronger than (, .)N 00 082
and that the
uniform noise is not stronger than
U (, .)00 25 when the embedding capacity
is less than 800 bits. Moreover, the
results in Fig. 11 indicate that greater
embedding capacity results in lower
0.14
0.12
0.1
0.08
0.06
0.04
0.02
robustness. This is also intuitively reasonable
since a greater embedding capacity
decreases the data redundancy.
In the field of data hiding in digital
images, three popular signal processing
attacks are used to check the robustness
of data hiding schemes: JPEG compression,
noising (Gaussian noise is usually
employed), and sampling [35]. For data
hiding in neural networks, JPEG compression
cannot be achieved. The noising
attack is already employed. To fit more
practical communication channels, the
robustness of the proposed scheme is also
checked using sampling. Since the decoding
networks are connected with
uji;
t
the parameters in uji;
t
training, and the extraction error is
checked. Denoting the ratio of removed
parameters as
a , the relationship between
a and extraction error e is shown
in Fig. 12, where
b 15= ., n 1= and
,
batch size is 50.
The results show that the proposed
scheme has good robustness against sampling.
For instance, when the capacity is
less than 5000 bits, the additional data
can be extracted correctly with only
approximately half of the parameters
in
u .ji;
t
Even when the capacity is
in the original network, a part of
are removed after
Capacity = 6,000 bits
Capacity = 5,500 bits
Capacity = 5,000 bits
Capacity = 4,500 bits
Capacity = 4,000 bits
6000 bits (the maximal capacity of the
proposed scheme), correct extraction can
be guaranteed with approximately 70%
of the parameters. As a result, the proposed
scheme is robust against sampling
to some extent.
Since the proposed scheme is robust
to some extent, it can be used for real
applications. As shown in Fig. 2, after the
additional data for multiple receivers are
embedded into CapsNets, the sender
transmits the obtained CapsNets to n
receivers. In addition, as mentioned in
Section I, the proposed scheme is practical
since the parameters of data decoding
networks are determined by embedding
keys instead of training. The parameters of
decoding networks can thus be obtained
with correct embedding keys. Therefore,
it is unnecessary to transmit the decoding
networks to the receivers.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
α
FIGURE 12 Extraction errors against sampling.
Max
Conv
Pooling
LRN
Conv
Max
Pooling
LRN
Conv
D. Universality on Other Networks
This paper proposes a data hiding scheme
for neural networks and achieves it using
CapsNets as an example. For other networks,
the proposed scheme is also effective.
To verify this, a group of experiments
are conducted using a simplified version
of AlexNet [36] on the MNIST dataset
(as shown in Fig. 13) and using ResNet18
[7] on ImageNet [36]. For the experiments
of ResNet18 on ImageNet, 10 categories
(1300 images in each category)
Max
Pooling
LRN
FC FC
Softmax
10
3 × 3, ReLU
Stride of 1
3 × 3
Stride of 2
FIGURE 13 Architecture of AlexNet for MNIST.
3 × 3, ReLU
Stride of 1
3 × 3
Stride of 2
3 × 3, ReLU
Stride of 1
3 × 3
Stride of 2
1,024
ReLU
1,024
ReLU
80 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | NOVEMBER 2021
Extraction Error e
IEEE Computational Intelligence Magazine - November 2021

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - November 2021
