IEEE Computational Intelligence Magazine - February 2023 - 77

FIGURE 11 Average quantitative evaluation of our SSN-CAEþIE with 10 state-of-the-art methods.
part. Note that the YCbCr color space is applied to fuse color
images due to the color distortion that may rise by some methods
[56], [57] when fusing directly on the RGB color space.
The first step is to use the proposed scheme to fuse MRI
images with the Y component of the PET or SPECT images.
Then, the fused results are used to replace the Y components
of the PET/SPECT images in the YCbCr space to obtain the
final color fusion image. Finally, the fused image is transfered
to RGB space to get the result.
For a better comparison, we set up green and red boxes
and enlarged them into close-ups. As shown in Fig. 9, the proposed
method can obtain satisfactory results on three different
categories of PET/MRI and SPECT/MRI image fusion. In
the red boxes and their close-ups of Set-6, the results of
EMFusion, MSMG, and CSMCA cannot retain the salient
information ofMR_T2. In addition, LRD exists color distortion
compared with other methods. In the green remarked
boxes ofSet-7, the fused results ofMSMG and ZL have salient
structural information ofMR_T2, but reduce the color intensity
in SPECT_Tc. Furthermore, as shown in the red boxes
and the zoomed-in regions of Set-8, EMFusion and ZL can
retain the functional information ofthe SPECT_Ti image, but
cause the loss of texture structure information of MR-T2 in
green boxes and the zoomed-in regions. Among them, our
method not only performs effectively on spectral information
but also contains a rich texture structure. The quantitative
evaluations of different image fusion methods are given in
Table II, a better fusion performance can be achieved by the
proposed method.
The quantitative results ofall the methods for the displayed
set images are shown in Fig. 10. The average quantitative
results of all the methods for the eight types ofmedical images
are shown in Fig. 11 and the ranking of the methods is shown
in Table I. Fig. 11 displays the average evaluation of our
framework with the other 10 state-of-the-art methods. The
method proposed in this paper has obvious advantages over
other methods.
C. Parameter Discussion
To further demonstrate the influence of in the information
exchange loss (Lie), we also conduct experiments with
varioussettingsof. We select different to train the
SSN-CAEþIE to minimize the LossMCIEN to produce different
fused results for 80 pairs of medical images and calculate
the average evaluation results in terms of all metrics.
Table III shows the Lie performances for our method
when is taken with different values. We can see that the
FSIM, UIQI, and NMI results reach the maximum value
when ¼ 0:7, and gradually decreases, before and after
¼ 0:7. Comprehensively, our method obtains the bestfused
results when ¼ 0:7. Hence, is finally selected as
0.7 in our case.
D.Ablation Study
We take the CT/MR-T2 test dataset as an example to perform
a series ofexperiments to validate the effectiveness ofthe
network architecture and loss functions in the proposed
method. Table IV shows the comparison fused results ofdifferent
architecture strategies and Table V shows the comparison
fused results ofdifferent loss strategies. Fig. 12 shows the comparison
of different strategies for the fusion contribution
estimation.
TABLE I Average rankings of different methods on all datasets for each evaluation metric.
METRIC
IFCNN
(2020)
FSIM "
11
10
10
DDCGAN
(2020)
10
UIQI " 811
QAB/F "
NMI "
AVERAGE 9.75
11
11
10.75
U2Fusion
(2020)
8
9
8
7
8
The top six methods are shown in RED BOLD fonts.
PMGI
(2020)
9
10
9
9
9.25
LRD
(2020)
6
3
7
3
4.75
CSMCA
(2015)
2
3
4
ZL
(2019)
7
4
5
MSMG
(2020)
4
5
6
PAPCNN
(2018)
5
EMFusion
(2021)
3
4 576 2
6
8
3.25
5.25
5.5
6.25
2
2
2.25
OURS
1
1
1
1
1
FEBRUARY 2023 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 77

IEEE Computational Intelligence Magazine - February 2023

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - February 2023