IEEE Computational Intelligence Magazine - May 2022 - 27

6
4
2
0.5
1.0
0 0.2 0.4
0.2
0.4
0.6 0.8 12 46 8 -10 -5 05 10
QED
(a)
SAS
(b)
LOGP
(c)
ZINC MO-DEL SO-DEL GB GB (DME) MO-PSO MO-PSO (DME)
FIGURE 7 Property distributions of results from methods on ZINC.
H. Comparison with Multi-Objective Particle
Swarm Optimization (MO-PSO)
A standard PSO is used in [55] in the latent space of a VAE by
linearly combining objectives of interests. Strictly speaking, it is
a SO method. The hyperparameter settings from [55] were utilized
in our experiments. From Table V and Figure 7, it can be
seen that the DEL methods outperform the MO-PSO methods,
and MO-PSO(DEM) obtained better results than that of
the MO-PSO in [55]. A true MO-PSO will be investigated in
the framework of data-model co-evolution.
V. Discussion and Conclusion
In this paper, we presented our prototypical DEL framework
where a fragment-based VAE is integrated such that evolutionary
exploration is conducted in the continuous latent representation
space rather than the discrete structural space. Our
intensive experiments show that DEL is able to generate novel
populations of molecules with improved properties, and outperforms
state-of-the-art benchmarks.
The major novelty of our idea is the substantial integration
of ECs with DGMs (rather than ECs and DGMs themselves)
through the latent space of neural generative models. This is
distinct from other EC approaches such as [21], [56]-[59]
which work in the original space and were not combined with
DGMs. Our DEL framework is simple and flexible, thus other
DGMs (we do not intend to promote FragVAE) and multiobjective
computational intelligence algorithms can be easily
embedded in it. Here we demonstrate a prototype of our idea
and show that it performs well. The recent work in [60] is
seemingly similar to our work. However, their work does not
take advantage of the latent representation space nor multiobjectivity.
Thus, the work in [60] is essentially very different
from DEL. Another work presented in [55] utilizes a multiobjective
swarm optimization algorithm (actually a weighted
sum of several objectives) in the latent space of VAE trained on
SMILES. The major difference between DEL and theirs is that
we fine-tune the DGM using new populations of samples with
better properties and show that this strategy does improve the
performance in comparison to EC with static DGM.
Besides the deep neural network training and inference, the
scalability of DEL is largely determined by the efficiency of the
non-dominated sort algorithm which has time complexity of
()
OKM2
[42] where K is the number of objectives and M is
the population size. The DEL experimented on the ZINC data
runs for nearly 3 hours for population size of 20K and about
14 hours for population size of 100K. To improve the efficiency
of DEL, more efficient ranking algorithms can be employed.
For example, the average ranking algorithm only has time
complexity ()
OK logMM [43] which could be used together
with diversity assessment metrics [61].
Applications of DEL are certainly not restricted to molecular
design. In future work, DEL will be tested on different datasets,
other design problems, and more specific applications.
Other types of DGMs as well as gradient- and non-gradient
based search strategies will be explored to further enhance
DEL. Particularly, geometric deep learning models which represent
molecules in 3D space [62], [63] will be investigated for
integration in DEL. New MO-BO algorithms for latent-space
based optimization need to be studied further to address issues
such as scalability, unknown invalid domains in latent space, and
the curse of dimensionality. Implementation of our method is
available upon request.
Acknowledgment
This research team was majorly supported by the Artificial
Intelligence for Design Challenge Program at the National
Research Council Canada. Y.L. was also funded by the Natural
Sciences and Engineering Research Council of Canada
(NSERC) Discovery Grant (RGPIN-2021-03879). K.G. was
meanwhile supported by the Vector Scholarship in AI awarded
by the Vector Institute. This article has supplementary downloadable
material available at https://doi.org/10.1109/
MCI.2022.3155308, provided by the authors.
References
[1] C. Lipinski, F. Lombardo, B. Dominy, and P. Feeney, " Experimental and computational
approaches to estimate solubility and permeability in drug discovery and development
settings, " Adv. Drug Delivery Rev., vol. 46, nos. 1-4, pp. 3-26, 2001, doi: 10.1016/
S0169-409X(00)00129-0.
MAY 2022 | IEEE COMPUTATIONAL INTELLIGENCE MAGAZINE 27
https://www.doi.org/10.1109/MCI.2022.3155308 https://www.doi.org/10.1109/MCI.2022.3155308
IEEE Computational Intelligence Magazine - May 2022

Table of Contents for the Digital Edition of IEEE Computational Intelligence Magazine - May 2022
