IEEE Consumer Electronics Magazine - September/October 2021 - 99

Figure 4. Training process of RR-LADP and CDP
when K ¼ 1; 000 and K
¼ 100. The solid lines
represent loss and the dotted lines represent d
( " ¼ 8; s ¼ 1).
average, and treats the training process of each
client as a black box. Then, its performance usually
fluctuates.
Comparison with CDP: To compare RR-LADP
and CDP, we track privacy budget in each mechanism
by calculating d. In fact, accumulated
knowledge inheritance pre makes the privacy
loss of each round increase rapidly. Saving privacy
budget of the same order of magnitude cannot
support more training rounds. Therefore,
careful allocation means efficient data usage.
Figure 4 tracks the loss and d as communication
rounds increased under two mechanisms. It
shows that RR-LADP allocated the privacy budget
more carefully while the loss of global model
drops as similar to CDP.
Table 1 shows the average accuracy of multiple
training and training time per round. RRLADP
achieves higher accuracy than CDP while
increasing the time cost. The accuracy of RRLADP
depends on the effects of RR and LADP,
which has been described in the separate
experiments. For CDP, the noise addition operation
for privacy appears only once on the server
Table 1. Accuracy and Time Spent in Each Round of RRLADP
and CDP ( " ¼ 8; s ¼ 1).
Clients Rounds
100
1,000
10,000
100
200
400
CDP
0.73 j 25.9 s
0.91 j 85.7 s
0.96 j 315.9 s
RR-LADP
0.80 j 26.5 s
0.92 j 89.2 s
0.97 j 336.1 s
The bold values in this tabel highlight that RR-LADP outperforms
CDPon accuracy.
Figure 5. Results on the accuracy and d for different
privacy budgets. The solid lines represent accuracy
and the dotted lines represent d
(s ¼ 1;K ¼ 1; 000;K
¼ 100).
side in each training round. However for RRLADP,
all clients in one training round should
perform the noise addition operation, bringing
the more time delay.
Factors affecting RR-LADP: The accuracy of the
final model is affected by multiple factors, such
as the batch size, learning rate, and other common
parameters in machine learning. We have
not struggled to find the best combination of
these parameters, but only discussed some significant
factors in RR-LADP, including number of
clients, privacy budget, and the noise.
The number of clients determines the training
data anddetermines the possibility of privacy leakage
by affecting the sampling probability. The
lower privacy loss in each roundmeansmore training
rounds and higher accuracy. At the same time,
the privacy bound, which is set to
jKBj , decreases
1
significantly if the number of clients increases.
Privacy budget " plays a pivotal role in RRLADP
(see Figure 5). It controls the training process
of in two ways. First, it determines the
response probability p, which affects the disturbance
in the RR mechanism. In fact, K
K p in (3)
represents the part selected by the server to participate
in training. The server infers the local
updates of a particular client easily if overlap ratio
is high. Second, after updating global model in
each round, we track d with the fixed " .The lower
" , the higher d, which means it is easier to reach
the boundary and the fewer rounds are allowed.
Our mechanism allows to control model performance
by choosing the value of s. The level
of noise added in the local updates can directly
affect the accuracy of the model. As shown in
September/October 2021
99
IEEE Consumer Electronics Magazine - September/October 2021

Table of Contents for the Digital Edition of IEEE Consumer Electronics Magazine - September/October 2021
