IEEE Computational Intelligence Magazine - February 2023 - 79

TABLE IV Quantitative comparison of fusion performance on
test datasets our method architecture and its degraded
versions.
VERSION CNN VIT SS IE FSIM " UIQI " QAB/F " NMIN "
Ver 1 ✗✓ ✓
Ver 2 ✓✗ ✓
Ver 3 ✓✓ ✗
Ver 4 ✓✓ ✓
Ours ✓✓ ✓
✓ 0.8161 0.6630 0.7361 0.9075
✓ 0.8263 0.6827 0.7893 1.1245
✓ 0.8264 0.6832 0.7881 1.1164
✗ 0.7955 0.6232 0.6305 0.7812
✓ 0.8272 0.6870 0.7874 1.1247
The average best and second results are shown in RED and Blue BOLD font.
TABLE V Comparison of fusion performance on test datasets
our method losses and its degraded versions.
VERSION CONTRASTIVE IE FSIM "
UIQI " QAB/F " NMIN "
Ver 1 ✗✓ 0.8252 0.6849 0.7978 1.1022
Ver 2 ✓✗ 0.8266 0.6839 0.7895 1.1448
Ours ✓✓ 0.8272 0.6870 0.7874 1.1247
The average best and second results are shown in RED and Blue BOLD font.
and 0.3435 on each metric respectively, over the version without
IE. The self-supervised strategy makes our framework
more adaptive to feature extraction, the version without SS
also presents worsen performances in contribution estimation
and each metric. Therefore, Vit, CNN, IE, and SS all provide
significant contributions to our method, while IE and CNN
are especially essential.
2) Ablation to Loss
Let LossCAE discard the contrastive loss for negative results
(LC), LossMCIEN discard the information exchange loss (Lie), to
demonstrate the effect of the contrastive loss and the information
exchange loss for the proposed method. We can see from
Table V that degraded versions present worsen performances
in terms ofFSIM and UIQI, compared with our method, and
that our version gets the second-best result in NMIN. As
shown in Fig. 12, versions without LC and Lie are not conducive
to estimating better fusion contributions. Therefore, the
LC and Lie in SSN-CAEþIE are both beneficial to improve
fusion performance.
V. Conclusion and Future Work
In this paper, a self-supervised network based on contrastive
auto-encoder and information exchange is presented for the
MMIF task which consists of a feature extractor and a
weighted contribution estimation network, i.e., CAE and
MCIEN. Source images are given into the CAE to extract the
non-redundant features of paired source images. CAE is comprised
ofVit and CNN to obtain the global and local information
of source images at the same time. Furthermore, a novel
contrastive loss is adopted to constrain the CAE extracts irredundant
features of the paired source images. Then, the
FIGURE 12 Quantitative ablation studies of our SSCAE-IE on different
versions for fusion contribution estimation of source images A, B. (a)
Source images; (b) SSN-CAEþIE; (c) No CNN; (d) No Vit; (e) No SS; (f)
No IE; (g) No Lie; (h) No LC.
extracted features are fed into the MCIEN to estimate the
contribution of paired source images. In addition, a weighted
fidelity loss and an information exchange loss are introduced
in the MCIEN to effectively train the SSN-CAEþIE. Compared
with nine state-of-art methods, the qualitative and
quantitative results verify the superiority of the proposed
method in terms of visual aspect and quantitative analysis. In
the future, we will pay more attention to three directions: 1)
Optimize each sub-task consisting of medical image fusion
tasks from different modalities through the core idea of metalearning,
achieving great universality via a few short datasets
[58]. 2) Combine the image registration [49], [50] and fusion
tasks to construct a joint framework. 3) Expand SSN-CAEþIE
to infrared-visible image fusion (e.g., differentiate the loss of
infrared-visible image fusion task in meta-learning) to verify
the universality and adaption of the proposed method due to
the similar essence between infrared-visible and MRI&PET
image fusion [31], [34].
Acknowledgment
This work was supported in part by the National Natural Science
Foundation of China under Grants 61966037 and
61833005; in part by the National Key Research and Development
Project of China under Grant 2020YFA0714301; in
part by China Postdoctoral Science Foundation under Grant
2017M621586; and in part by the Yunnan Provincial Department
of Education Science Foundation under Grant
2022Y011.
FEBRUARY 2023 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 79
IEEE Computational Intelligence Magazine - February 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2023
