IEEE Computational Intelligence Magazine - May 2022 - 24

Hypervolume and non-dominated sorting are
informative measures for comparison of multiobjective
solutions and were used in our comparison.
data in each generation, forms new child latent codes using
the blending linear crossover method, and fixes the population
size to 20K. Variants were created by (1) disabling the property
predictor, (2) disabling fine-tuning, (3) using the discrete crossover
method, and (4) allowing a much larger population size
(100K). While it is difficult to compare these variants in terms
of validity, novelty, and diversity of population samples (see
Figure S22), it turns out that hypervolume and non-dominated
sorting are informative measures for comparison of multiobjective
solutions and were used in our comparison. Table
SIII in the Supplementary Materials indicates that DEL with
the property predictor outperforms the variant without it.
Table SIV shows that FragVAE fine-tuning in DEL can help
obtain better Pareto fronts. Table SV implies that both linear
and discrete crossover operations behave well in DEL. Also,
DEL with larger population size can form better Pareto fronts
(see Table SVI). Hypervolume and non-dominated sorting
were also applied to compare the quality of Pareto fronts
obtained using different values of b on DEL. It is found that
slightly better results were obtained when setting b to 0.1
than the other values mentioned in Section IV-B, but the integrated
Pareto front consists of samples relatively even from all
settings (Table SVII).
To investigate whether solutions of DEL using different values
of hyperparameters are similar, we further investigated the similarity
(in terms of Tanimoto distance) of generated high-quality
TABLE II Comparison of some fragments' frequencies in
DEL's first Pareto front and in the training data. The DEL
results here were obtained by setting b = 0.1.
FRAGMENT
[NH2+]CC
CC=C
CCCOC
c1cc(OC)ccc1OC
[NH+](C)CC
c1cn(C)nc1C
FRAGMENT
NC(*)=O
N(CC)CC
C(C)(C)CC
N1CCN(CC)CC1
C(CC)CO
CCO*
FREQ. IN
DEL
0.3655
0.1846
0.0711
0.0701
0.0383
0.0279
FREQ. IN
DEL
0.1684
0.0884
0.0643
0.0474
0.0374
0.0170
FREQ. IN
ZINC
0.0197
0.0175
0.0038
0.0030
0.0117
0.0041
FREQ. IN
PCBA
0.0138
0.0158
0.0033
0.0029
0.0003
0.0033
FREQ.
RATIO
18.5405
10.5168
18.8891
23.3443
3.2796
6.7249
FREQ.
RATIO
12.1714
5.6055
19.2193
16.6286
111.1206
5.1303
novel molecules among different DEL experiments
using different values of b (and ).a Suppose
Pi
m P ,
()
i
most similar molecule m P()
k ! we can find its
j
j
to Pj
of size K and Pj of size Kl are a pair
i
or Pareto fronts from two DEL experiments.
For any molecule
kl ) ! in terms of
Tanimoto similarity (in range [0, 1]). Tanimoto distance is defined
as 1 - Tanimoto similarity. The distance from Pi
For each closest molecule pair (, ),mm
()
i
k
to Pi
ij = 1 K Rk=1Tanimoto_Distance(, )l )
()
kl )
ed as TD ( ,)(.PP /) K
j
mmk
()i
j
k
the corresponding
absolute differences in properties QED, SAS, and
logP were also recorded, and finally averaged over the front size
K. Similarly, the distance from Pj
ji = 1
TD ( ,) (/ )( ,). By
the asymmetric nature between fronts, usually TD PP !
TD(, ).PP The comparison results can be seen in Table III, in
PP Km m )
l Rkl =1Tanimoto Distance
Kl
-
ji
which it can be concluded that the novel molecules found using
different hyperparameters are mildly similar, and the corresponding
property values are close.
E. Comparison with Multi-Objective
Bayesian Optimization (MO-BO)
DEL was compared with two MO-BO methods: q-Pareto
Efficient Global Optimization (qParEGO) and q-Expected
Hypervolume Improvement (qEHVI) [53]. These MO-BO
methods were applied in the latent space of FragVAE trained in
the first generation of DEL using all training samples. Hyperparameter
settings of these algorithms are listed in Section S.5
of the Supplementary Materials. Figure S23 shows the hypervolumes
of both algorithms and the quasi-random baseline
which selects candidates from a scrambled Sobol sequence
along iterations. It can be seen that qParEGO and qEHVI
work better with
b 00 . 1=
than that with b 01= ..
To qualitatively compare DEL with the MO-BO algorithms,
the first five Pareto fronts obtained using DEL and the
last five iterations obtained using qParEGO and qEHVI are
visualized in Figure 6 and Figure S24. Three-D animations of
these results can be found in the Supplementary Materials. One
can see that the solutions from qParEGO and qEHVI are
behind the Pareto fronts from DEL. To quantitatively compare
DEL with qParEGO and qEHVI, we recorded the hypervolumes
of individual first Pareto fronts obtained by these methods,
and also conducted non-dominated sorting on the
combination of the first Pareto front of DEL and the last six
iterations of qParEGO and qEHVI and report the results in
Table IV. It shows that the hypervolumes of DEL are much
larger than those of MO-BO methods, and all solutions from
the DEL Pareto front stay in the new integrated Pareto front,
while almost all solutions from qParEGO and qEHVI are
behind the integrated Pareto front. Furthermore, as indicated in
Table SXII, DEL runs more efficiently than these MO-BO
algorithms, even though the population size (20K) of DEL is
much larger than the solution size (8) in each iteration of
MO-BO algorithms. It has been a well-known challenge to
scale up BO algorithms.
24 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | MAY 2022
j
kl
()i
k
( ,)
ij
is calculated as
()
is calculat()
IEEE Computational Intelligence Magazine - May 2022

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2022
