IEEE Computational Intelligence Magazine - May 2022 - 20

The property distributions of generated samples can
be important indicators of the proximity of generated
samples to actual samples.
zm
t
bution: t ml,z N (, ).01
is offset by a value drawn from the standard Normal distri+
D.
Forming New Population
Ideally, excellent and diverse populations need to be maintained.
After possible mutation operations, all M genotypic coding
vectors will pass through the decoder of the DGM to
produce phenotypic samples. All valid samples (supposed in set
P )t 1+
t
(denoted by P )t
P ).t 1+
will be kept to merge with the previous population
to produce a new generation (denoted by
t
To implement it, all samples in PP are sorted
tt1+ +
based on their non-domination ranks first and then on their
crowding distances. Finally, only the top M samples are taken
from them to form the new population.
E. Pretraining and Fine-Tuning FragVAE
The neural generative model FragVAE is initially trained using
all available training samples organized in mini-batches at the
very beginning of the DEL process. Since FragVAE learns from
scratch in the pretraining, the values of b and a remain constantly
small (e.g.,
b 01 .= or 0.01 and a 1 )= such that the
reconstruction error term (the most difficult term to be minimized)
in the loss function can receive more stress (see justifications
in Section III-A). The initial learning rate is gradually
reduced following a schedule (e.g., reduction at rate 0.8 for every
4 epochs). Furthermore, after forming a new population, FragVAE
can be fine-tuned using samples in the new population. For
each fine-tuning process, since the population size M is less than
(or equal to) the original training set size, FragVAE is fine-tuned
for fewer epochs and uses a faster learning rate annealing schedule
than the pretraining process. Especially in the second generation,
i.e., the first fine-tuning, the values of b and a either
remain the same as these in the pretraining or increase gradually
to larger values such that the KL divergence and property regression
terms are subject to more pressure to improve.
IV. Experiments
The performance of DEL was investigated on the ZINC [46],
[47] and PCBA [48] datasets. The authors of [33] processed a
subset of ZINC and PCBA and obtained 227,945 and 383,790
molecules respectively with two or more fragments. Our
experiment also used the processed data from MOSES [49] to
compare DEL with the baselines available in MOSES. The
empirical performance of FragVAE and DEL was comprehensively
investigated. Three properties (QED: quantitative estimation
of drug-likeness, SAS: synthetic accessibility score, and
logP: water-octanol partition coefficient) are selected as objectives
in DEL. Molecules with large QED, low SAS, and small
logP values are prioritized. Incorporation of other properties
20 IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE | MAY 2022
(e.g., binding affinity, structure-property relationship,
and ADME - absorption, distribution,
metabolism, and excretion) will be
considered in future work. QED, a scalarization
of eight molecular properties (including
logP) [50], is an adequate initial screening step
for drug candidates. As we focus on specific applications, specific
properties can be selected for subsequent screening. Thus,
explicit usage of logP as one objective in DEL can help better
assess lipophilicity, a key factor in drug design for some diseases,
e.g., kidney and heart problems.
A. Evaluation of FragVAE
As FragVAE is a significant modification of the original model
used in [33], we investigated the impact of b value on the performance
of FragVAE in terms of loss function values through
Figure S2 (see Supplementary Materials). Other hyperparameter
values can be found in the Supplementary Materials. One
can see that a large b value can quickly reduce the KL loss to
near zero, leading to stagnant reductions of reconstruction error
and property regression error, which is an instance of the notorious
posterior collapse problem [51], because it is much easier
to reduce the KL divergence than the reconstruction error in
complex sequence modeling. Using a suitably small value of b
would allow for the continuous decrease of the reconstruction
error and property regression error. This observation is consistent
IEEE Computational Intelligence Magazine - May 2022

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2022
