IEEE Computational Intelligence Magazine - February 2023 - 74

tune the CAE at the same time. The loss for MCIEN is
defined as follows:
LossMCIEN ¼ LwfLie
FIGURE 7 Contrastive pairs in CAE. A is a positive result, B and B0are
the negative results for A0, vice versa for B0, B is a positive result, A and
A0 are the negative results. The red solid and black dashed arrows
denote the direction we want to push close or apart by a contrastive
loss.
contrastive learning, we construct the positive result hþ 2
Rmn and the negative results h
iiiiiii 2 Rmn; i ¼ 1; 2 for the
reconstructed image. As shown in Fig. 7, A is a positive result
(hþÞ, B and B0are the negative results (h
1 ; h
1 ; h
2 ). The aim is to push the reconstructed
2 ) for A0; vice
versa for B0, B is a positive result (hþÞ, A and A0 are the negative
results (h
images close to their source images, and apart them from other
reconstructed images and source images so that the unique features
can be extracted. To avoid pixel error, we innovatively
adopt the L2-norm jj jj2
2 to measure the distance between the
reconstructed images and the positive and negative results
instead of the traditional InfoNCE loss [44]. The formulation
is shown as follows:
LossCAE ¼ LA þ LB
(10)
where LA is a contrastive loss for the reconstruction ofA; LB is
a contrastive loss for the reconstruction ofB and LossCAE consists
ofLA and LB.
LA ¼ A0Ajj jj2
jj2
jj2
LB ¼ B0 Bjj jj2
which is simplified as:
LossCAE ¼jjA0Ajj2
2gð A0Bjj jj2
2gð B0Ajj jj2
2
2 þjjB0 Bjj2
2
gðjjA0 Bjj2
þ 2jjA0 B0jj2
2
ðjjA0Ajj2
2 þjjB0Bjj2
2 þjjB0Ajj2
2
(13)
2Þ is exactly the loss offidelity in the
traditional auto-encoder. Therefore, LossCAE can also be formulated
as follows:
LossCAE ¼ LF gLC
where LF ¼jjA0Ajj2
2 þjjB0 Bjj2
(14)
2 is the fidelity loss, LC
denotes contrastive loss for negative results and g is constant to
balance the importance of the negative results in our contrastive
loss, which is empirically assumed as 0.1.
2) Training for MCIEN
For the MCIEN, a weighted fidelity loss and an information
exchange loss function are adopted to train it and reverse fine2þ
A0 B0jj2 (11Þ
2þ B0A0jj (12)
1
Lie ¼
HW
ðA; iÞ " A
(15)
where Lwf is the weighted fidelity loss, whereas Lie denotes the
information exchange loss of the multi-convolutional
exchanged information between a pair ofsource images. is a
constant to balance the importance of the information
exchange loss in the loss ofMCIEN.
Lwf ¼ WA A0AðÞ
jj2
jj2 þ WB B0 BðÞ 2jj2
jj (16)
WA and WB represent the weights of image A and image B,
A0 and B0 are the reconstructed images, WAþWB¼ 1.
X3
i ¼ 1
" B
ðÞA; i " A
ðÞB; i
2
2
(17)
whereHW is the product ofthe length and width ofthe source
image, ( " B
ðB; iÞÞ denotes the exchanged information distance
between A and B in the i-th convolution space. The more
sufficient the information exchange between two objects is, the
more accurately and easily we can measure their contributions
to a certain cooperative task; it is expected that the information
exchange among pairs of features should be sufficient as far as
possible. We should maximize the distance among pairs of
exchanged information at different convolution spaces.
3)The Weight ofthe Fidelity Item
MCIEN measures the importance of each pixel by a hybrid
method ofcalculating the salient features and the local contrast
features of the image. Specifically, WA and WB are formulated,
respectively, as follows.
Ua
WA ¼
WB ¼
ðUa
ðUa
A þVb
A þVb
Ua
A þVb
AÞþ Ua
B þVb
AÞþ Ua
(18Þ
B
A
B þVb
B þVb
B
(19)
B
where UA and UB are respectively the salient features ofA and
B, V represents the local contrast feature calculation of the
weight of the fidelity item; and a and b are two adjustable
constants to balance the importance related to the salient features
and local contrast features in our hybrid measurement,
which is empirically assumed as 1 and 40 in our case.
a) Salientfeature: Pixels ofhigh salience in an image are usually
the features that draw the most attention. We multiply the
saliency and brightness of the image as part of the weight of
the fidelity term in the construction of the loss function, the
saliency is calculated by LC-based saliency [45].
U ¼ S L
(20)
where S and L represent the salient detection and brightness of
the image respectively.
74 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | FEBRUARY 2023
IEEE Computational Intelligence Magazine - February 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2023
