Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_5580_Библиотеки_им_академика_М_И_Перельмана
.pdf
112 M. Yamada and M. Sugiyama
https://t.me/med1917
Fig. 6.8 Reassembling two molecules with nodes (a)oredges(b). (a) shows merging with nodes
labeled as C. (b) shows merging with edges in rings. In this examples, reassembled molecules are
sanitized to be valid molecules
6.3.3 Finding Candidate Subgraphs by Reinforcement
Learning
Our approach of graph generation is an autoregressive manner so that the target
molecule is built up through assembling subgraphs as shown in Fig. 6.9. Reinforcement learning is an effective approach in such a task to find subgraphs that will lead
to desirable molecules. Other potential choice of a searching method is Monte Carlo
tree search (MCTS), which is a search method that combines tree search with random
sampling [
massive and achieved huge success in various fields such as the game of Go [
the planning of chemical syntheses [
can be reusable, we adopt the reinforcement learning. If the environment for molecular graph generation is set up, any algorithms, such as deep Q-network, actor critic,
proximal policy optimization (PPO), etc., can be utilized. We adopt PPO that can be
mainly used for continuous control, likewise GCPN and GraphAF. The environment
for generating molecules is defined as below.
vectors by Mol2vec [
designed for molecular substructures obtained through Morgan fingerprints. In
Mol2vec, each node feature in a graph is reduced via sum pooling into a graph
feature vector, which includes the graph topological information.
from subgraph mining algorithm. Each action
of the likelihood based on the state value function
reward.
target properties satisfies the additive compositionality of subgraphs. Therefore, if
25]. MCTS has been applied to a number of tasks when the search space is
] and
26
]. Considering that the trained policy model
18
States space. A molecular graph Gt is embedded into 300-dimensional feature
], which is based on a pretrained word2vec [28] model
27
Action space. Action space A is equivalent to the set of subgraphs enumerated
a ∈ A is sampled according to softmax
SV (st ) and the expected cumulative
Reward function. To generate molecules with target properties, we assume that
6 Subgraph-Based Molecular Graph Generation 113
https://t.me/med1917
Fig. 6.9 The diagram of molecular generation in reassembling step. Mined subgraph structures are
selected based on the policy network
based on the highest score of a target property computed from a reward function
πθ . After reassembling molecules, a new graph is selected
some subgraph structure is not related to the target property, it is not selected as a
building block. This assumption can be simply represented in the following strategy:
If the difference of rewards at a time step t is negative, we stop the trial. The reward
rt is computed by a scoring function f (Gt) at time step t. If the reward is below a
certain threshold of the score, further search will be stopped.
6.4 Benchmark for Molecular Generation
As for molecular graph generation benchmarks, GuacaMol [29] and MOSES [30]are
proposed to evaluate the performance of generative models and each algorithm. In
GuacaMol, there are mainly two types of benchmarks: distribution-learning benchmarks and goal-directed benchmarks. We use GuacaMol benchmark to empirically
examine the effectiveness of our proposed framework MOLDR while comparing to
the state-of-the-art molecular generation methods. In particular, we have examined
the two standard criteria, the penalized
of generated molecules. Furthermore, we benchmark the rediscovery molecules
using GuacaMol benchmark dataset. The detailed results and experiment settings are
described in our paper [
2]. In this book, we introduce the result of standard criteria
of QED and penalized logP, and rediscovery molecules benchmark. MOLDR can
generate intermediate molecules while optimizing target properties.
Target properties. As target chemical properties, we employ scores of the penal-
log P introduced in [7
ized
] and Quantitative Estimation of Drug likeness (QED) [31].
These values are widely used as a benchmark for the task of a molecular generation.
The penalized
log P is a logarithm of the octanol-water partition coefficient with
restrictions on the ring size and synthetic accessibility [
Rediscovery molecules. We examined whether or not MOLDR can reconstruct
target molecules such as drugs, and can generate molecules exceeding some threshold
log P and the drug-likeness score QED,
32].
114 M. Yamada and M. Sugiyama
https://t.me/med1917
of similarity between molecules, not just generating molecules with chemical properties. In such cases, we chose Celecoxib, Troglitazone and Thiothixene as rediscovery
benchmarks, Aripiprazole as similarity benchmark, and Ranolazine and Osimertinib
as multi-properties optimization (MPO) benchmarks.
6.4.1 Results of Log P and QED
Table 6.2 shows top 3 generated molecules according to property scores of the penal-
log P and QED. Scores of other methods come from [11, 12
ized
similar to the method of JT-VAE as both methods use junction trees, while MOLDR
outperforms both scores. The
log P is related to lipophilicity (fat solubility) and
hydrophilicity (water solubility) of a molecule. Hence, if the nodes in a generated
molecule include more carbon atoms (C) and less imide (=NH) or hydroxyl groups (OH), the resulting
carbon atoms is, the higher the
log P becomes higher. This means that, the larger the number of the
log P value is. While we show penalized log P scores
where the ring size and synthetic accessibility are penalized as shown in Table.
if we compute the non-penalized
log P score, it tends to be higher and the score is
22.47 for the top molecule of MOLDR. In the case of penalized
an approach of greedy search such as selecting only C can be enough to maximize
the score, because the calculation of the
log P score consists of an additive condition.
However, it is shown that MOLDR can explore and exploit molecules with perfectly
optimizing the penalized
log P without such a trick.
In contrast, the QED score is empirically derived from the combination of various
chemical properties and chemical structures. Hence it is not straightforward to maximize QED unlike the case of
log P and is more difficult. MOLDR outperforms the
score of JT-VAE, top-1 and -2 molecules generated by GCPN, and achieve the same
score of GraphAF. In order to increase the QED score, generated molecules need to
]. MOLDR is
6.2,
log P optimization,
Table 6.2 Comparison of the top 3 property scores of generated molecules. Scores for ORGAN,
JT-VAE, and GCPN are from [
Method Penalized log P QED
1st 2nd 3rd Validity
ZINC 4.52 4.30 4.23 100.0 0.948 0.948 0.948 100.0
ORGAN 3.63 3.49 3.44 0.4 0.838 0.814 0.814 2.2
JT-VAE 5.30 4.93 4.49 100.0 0.925 0.911 0.910 100.0
GCPN 7.98 7.85 7.80 100.0 0.948 0.947 0.946 100.0
GraphAF 12.23 11.29 11.05 100.0 0.948 0.948 0.947 100.0
MOLDR
(Our)
12.46 12.20 12.04 100.0 0.948 0.948 0.947 100.0
11
]and [12] on the ZINC dataset
(%)
1st 2nd 3rd Validity
(%)
6 Subgraph-Based Molecular Graph Generation 115
https://t.me/med1917
Fig. 6.10 Generated molecules based on ZINC dataset by MOLDR+PPO with penalized log P and
QED scores
follow the strict restriction of structures. Figure 6.10 illustrates examples of generated molecules with optimization of the penalized
MOLDR. In penalized
log P optimization, the molecular size is larger and molecules
log P and QED, respectively, by
include the large number of C. In QED optimization, the size of molecule is smaller
than the case of
log P optimization, and they have subgraphs that contribute to the
QED.
Theoretically, QED can have a value between 0 and 1, while if we use the default
weights w of RDKit for QED computation, the theoretical maximum value of QED
is 0.948449. Due to our approach of reassembling subgraphs, generated molecules
have high similarity between molecules and have the s ame components in the ZINC
database with slightly different positions. The molecule with the highest QED score
is already included in ZINC, which is the nearest to the optimal QED score.
6.4.2 Results on GuacaMol Benchmark
Table
6.3 shows the result of rediscovery benchmarks. MOLDR can generate the
target molecules with high accuracy, whose scores are competitive with SMILES
LSTM iteratively fine-tuned with hill-climbing, and Graph GA [
notable difference between those models is that our model can visualize the generating process of molecules, not just generating the target molecule. Although in
SMILES LSTM, intermediate molecules are evaluated from the state value function,
and the SMILES character is selected based on the state, it is not easy to interpret
why the character is vital at a particular time step, especially when generating a ring.
In contrast, in MOLDR, building blocks are directly selected, and the sub-structure
affects the target directly. Generating process is shown in Figs.
the generation performance of MOLDR depends on the building blocks obtained
from graph mining, in practical applications, it is important to prepare an appropriate
dataset and set an appropriate minimum support based on a priori knowledge. In the
task of generating molecules with multi-property optimzation (MPO), the generated
molecules optimizing Osimertinib MPO and Ranolazine MPO are shown in Fig.
]. The most
33
6.11 and 6.12. Since
6.13,
116 M. Yamada and M. Sugiyama
https://t.me/med1917
and our MOLDR method can generate molecules similar to those generated by Graph
GA and SMILES LSTM.
Table 6.3 Goal directed benchmarks
Benchmark Best in dataset SMILES LSTM Graph GA MOLDR
Celecoxib rediscovery 0.505 1.000 1.000 1.000
Troglitazone rediscovery 0.419 1.000 1.000 1.000
Aripiprazole similarity 0.595 1.000 1.000 1.000
Osimertinib MPO 0.839 0.907 0.953 0.898
Ranolazine MPO 0.792 0.855 0.920 0.864
Fig. 6.11 Generating process on Celecoxib rediscovery
Fig. 6.12 Generating process of Troglitazone rediscovery. The number under the molecules denotes
the similarity score between a generated molecule and target. MOLDR can generate Troglitazone
in 8 steps
6 Subgraph-Based Molecular Graph Generation 117
https://t.me/med1917
Fig. 6.13 The result of MPO benchmark. Top 30 molecules sampled from MOLDR
6.5 Conclusion and Future Outlook
We have reviewed various methods for molecular generation and presented our algorithm MOLDR that mines subgraph structures efficiently and reassembles them
into new molecules. Our framework is related to Junction Tree VAE, while ours
reassembles subgraphs directly instead of using a generative model. Our approach is
general, hence it can be also applied to any graph-related generation problems. As for
future outlook, molecular generation via reinforcement learning should be applied
together with the autonomous experimentation. However, the current reinforcement
learning approaches to generating molecules need massive training process to obtain
molecules with desired properties, and many graph neural network models for embedding molecules are too black-boxed to judge whether the generated molecules are
truly optimized or not. Therefore, interpretable graph machine learning models (or
reward functions), such as neuro-symbolic approaches, need to be studied further.
Acknowledgements This work was supported by JST, CREST Grant Number JPMJCR22D3,
Japan, and JSPS KAKENHI Grant Number JP21H03503.
Data and Software Availability The open-sourced codes and data for MOLDR are available at
https://github.com/Masatsugar/graph-decomposition-reassembling.
References
1. Polishchuk PG, Madzhidov TI, Varnek A (2013) Estimation of the size of drug-like chemical
space based on gdb-17 data. Journal of Computer-Aided Molecular Design 27(8):675–679
2. Yamada M, Sugiyama M (2023) Molecular graph generation by decomposition and reassembling. ACS Omega 8(22):19575–19586,
3. Weininger D (1988) SMILES, a chemical language and information system. 1. introduction
to methodology and encoding rules. Journal of Chemical Information and Computer Sciences
28(1):31–36
https://doi.org/10.1021/acsomega.3c01078
118 M. Yamada and M. Sugiyama
https://t.me/med1917
4. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780,
https://doi.org/10.1162/neco.1997.9.8.1735
5. Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural
networks on sequence modeling.
6. Gómez-Bombarelli R, Wei JN, Duvenaud D, Hernández-Lobato JM, Sánchez-Lengeling B,
Sheberla D, Aguilera-Iparraguirre J, Hirzel TD, Adams RP, Aspuru-Guzik A (2018) Automatic
chemical design using a data-driven continuous representation of molecules. ACS Central
Science 4(2):268–276,
7. Kusner MJ, Paige B, Hernández-Lobato JM (2017) Grammar variational autoencoder. arXiv:
1703.01925https://doi.org/10.48550/arXiv.1703.01925
8. Yang X, Zhang J, Yoshizoe K, Terayama K, Tsuda K (2017) ChemTS: an efficient python
library for de novo molecular generation. Science and Technology of Advanced Materials
18(1):972–976,
9. Yang X, Aasawat T, Yoshizoe K (2021) Practical massively parallel monte-carlo tree search
applied to molecular design. In: International Conference on Learning Representations,
openreview.net/forum?id=6k7VdojAIK
10. Olivecrona M, Blaschke T, Engkvist O, Chen H (2017) Molecular de-novo design through
deep reinforcement learning. Journal of Cheminformatics 9(48):1–14,
s13321-017-0235-x
11. You J, Liu B, Ying Z, Pande V, Leskovec J (2018) Graph convolutional policy network for goaldirected molecular graph generation. In: Proceedings of the 32nd International Conference on
Neural Information Processing Systems, pp 6412–6422,
files/paper/2018/file/d60678e8f2ba9c540798ebbde31177e8-Paper.pdf
12. Shi C, Xu M, Zhu Z, Zhang W, Zhang M, Tang J (2020) GraphAF: a flow-based autoregressive model for molecular graph generation. In: International Conference on Learning
Representations,
13. Jin W, Barzilay R, Jaakkola T (2018) Junction tree variational autoencoder for molecular graph
generation. In: Dy JG, Krause A (eds) Proceedings of the 35th International Conference on
Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10–15, 2018,
PMLR, Proceedings of Machine Learning Research, vol 80, pp 2328–2337,
mlr.press/v80/jin18a.html
14. Takeda S, Hama T, Hsu HH, Piunova VA, Zubarev D, Sanders DP, Pitera JW, Kogoh M, Hongo
T, Cheng Y, Bocanett W, Nakashika H, Fujita A, Tsuchiya Y, Hino K, Yano K, Hirose S, Toda
H, Orii Y, Nakano D (2020) Molecular inverse-design platform for material industries. In:
Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery;
Data Mining, pp 2961–2969,
15. McKay BD (1998) Isomorph-free exhaustive generation. Journal of Algorithms 26(2):306–324,
https://doi.org/10.1006/jagm.1997.0898
16. Stephen HG, Andrew RJ (2009) Mckay’s canonical graph labeling algorithm. In Communicating Mathematics 479:99–111
17. Jin W, Barzilay D, Jaakkola T (2020) Multi-objective molecule generation using interpretable
substructures. In: III HD, Singh A (eds) Proceedings of the 37th International Conference on
Machine Learning, PMLR, Proceedings of Machine Learning Research, vol 119, pp 4849–
4859,
18. Segler MHS, Preuss M, Waller MP (2018) Planning chemical syntheses with deep neural
19. Geng Z, Xie S, Xia Y, Wu L, Qin T, Wang J, Zhang Y, Wu F, Liu TY (2023) De novo molec-
20. Zaki MJ, Meira W Jr (2014) Data Mining and Analysis: Fundamental Concepts and Algorithms.
21. Xifeng Yan JH (2002) gspan: Graph-based substructure pattern mining. International Confer-
https://proceedings.mlr.press/v119/jin20b.html
networks and symbolic ai. Nature 555(7698):604–610,
ular generation via connection-aware motif mining. In: International Conference on Learning
Representations,
Cambridge University Press
ence on Data Mining pp 721–724,
https://doi.org/10.1021/acscentsci.7b00572
https://doi.org/10.1080/14686996.2017.1401424
https://openreview.net/forum?id=S1esMkHYPr
https://openreview.net/forum?id=Q_Jexl8-qDi
https://doi.org/10.48550/arXiv.1412.3555
https://
https://doi.org/10.1186/
https://proceedings.neurips.cc/paper_
http://proceedings.
https://doi.org/10.1145/3394486.3403346
https://doi.org/10.1038/nature25978
https://doi.org/10.1109/ICDM.2002.1184038
6 Subgraph-Based Molecular Graph Generation 119
https://t.me/med1917
22. Houbraken M, Demeyer S, Michoel T, Audenaert P, Colle D, Pickavet M (2014) The indexbased subgraph matching algorithm with general symmetries (ismags): Exploiting symmetry
for faster subgraph enumeration. Plos One 9(5):1–15,
7896
23. Segler MHS, Kogej T, Tyrchan C, Waller MP (2018) Generating focused molecule libraries
for drug discovery with recurrent neural networks. ACS central science 4(1):120–131,
doi.org/10.1021/acscentsci.7b00512
24. Li Y, Vinyals O, Dyer C, Pascanu R, Battaglia P (2018) Learning deep generative models of
graphs.
25. Browne CB, Powley E, Whitehouse D, Lucas SM, Cowling PI, Rohlfshagen P, Tavener S,
26. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J,
27. Jaeger S, Fulle S, Turk S (2018) Mol2vec: Unsupervised machine learning approach with
28. Mikolov T, Sutskever I, Chen K, Corrado G, Dean J (2013) Distributed representations of words
29. Brown N, Fiscato M, Segler MH, Vaucher AC (2019) Guacamol: Benchmarking models for
30. MOSES Polykovskiy D, Zhebrak A, Sanchez-Lengeling B, Golovanov S, Tatanov O, Belyaev
31. Bickerton GR, Paolini GV, Besnard J, Muresan S, Hopkins AL (2012) Quantifying the chemical
32. Ertl P, Schuffenhauer A (2009) Estimation of synthetic accessibility score of drug-like
33. Jensen JH (2019) A graph-based genetic algorithm and generative model/monte carlo tree
https://openreview.net/forum?id=Hy1d-ebAb
Perez D, Samothrakis S, Colton S (2012) A survey of monte carlo tree search methods. IEEE
Transactions on Computational Intelligence and AI in Games 4(1):1–43,
1109/TCIAIG.2012.2186810
Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N,
Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D (2016) Mastering
the game of go with deep neural networks and tree search. Nature 529(7587):484–489,
doi.org/10.1038/nature16961
chemical intuition. Journal of Chemical Information and Modeling 58(1):27–35,
org/10.1021/acs.jcim.7b00616
and phrases and their compositionality. In: Proceedings of the 26th International Conference
on Neural Information Processing Systems - Volume 2, Curran Associates Inc., Red Hook, NY,
USA, NIPS’13, pp 3111–3119
de novo molecular design. Journal of Chemical Information and Modeling 59(3):1096–1108,
https://doi.org/10.1021/acs.jcim.8b00839
S, Kurbanov R, Artamonov A, Aladinskiy V, Veselov M, Kadurin A, Johansson S, Chen H,
Nikolenko S, Aspuru-Guzik A, Zhavoronkov A (2020) Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models. Frontiers in Pharmacology 11,
doi.org/10.3389/fphar.2020.565644
beauty of drugs. Nature Chemistry 4(2):90–98,
molecules based on molecular complexity and fragment contributions. Journal of Cheminformatics 1(1):8,
search for the exploration of chemical space. Chemical Science 10:3567–3572,
10.1039/C8SC05372C
https://doi.org/10.1186/1758-2946-1-8
https://doi.org/10.1371/journal.pone.009
https://
https://doi.org/10.
https://
https://doi.
https://
https://doi.org/10.1038/nchem.1243
https://doi.org/
Chapter 7
https://t.me/med1917
Language Models in Molecular Discovery
Nikita Janakarajan, Tim Erdmann, Sarath Swaminathan, Teodoro Laino,
and Jannis Born
7.1 Introduction
The Turing test—envisioned in 1950 as a machine’s ability to simulate human
behavior to the extent of indiscernibility—served for decades as the holy grail of
artificial intelligence (AI). In recent years, language models (LMs) have demonstrated an astonishing ability to understand and generate human-like text [
progress has led to a shift in the perception of the Turing test, from being a heavily
debated and largely unsolvable challenge to now being silently acknowledged as
more achievable. Machine learning (ML) in general and LMs in particular hold the
potential to profoundly accelerate the molecular discovery cycle (see Fig.
we explore applications of LMs to chemical design tasks.
Despite technological advances constantly reshaping our understanding of
biochemical processes, the chemical industry persistently faces escalating resource
costs of up to 10 years and 3 billion dollars per new market release [
cacy of the problem is typically attested by an exorbitant attrition rate in in vitro
screenings [
serendipity [
3], the sheer size of the chemical space [4], and the frequency of
5].
Although LMs were originally developed for natural language, they have shown
compelling results in scientific discovery settings when applied to “scientific
], de novo design of small molecules [7],
languages”, e.g., in protein folding [
8
peptides [
] or polymers [9]. But what exactly is a language model? By defini-
6
tion, it is any ML model that consumes a sequence of text chunks (so-called tokens)
1]. This
7.1). Here,
]. The intri-
2
N. Janakarajan · T. Laino · J. Born (B)
IBM Research Europe, Zurich, Switzerland
e-mail: jab@zurich.ibm.com
T. Erdmann · S. Swaminathan
IBM Research Almaden, San Jose, CA, USA
N. Janakarajan
Department of Computer Science, ETH Zurich, Zurich, Switzerland
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024
H. Satoh et al. (eds.), Drug Development Supported by Informatics,
https://doi.org/10.1007/978-981-97-4828-0_7
121
122 N. Janakarajan et al.
https://t.me/med1917
Fig. 7.1 A comparison of molecular discovery workflows: a classic approach, where each hypothesis (a.k.a. molecule) requires a new experimental cycle. b Accelerated molecular discovery cycle
with machine-generated hypotheses and assisted validation, enabling simultaneous generation and
testing of numerous molecules
and is capable of reasoning about the content of the sequence. Since each token is
essentially a vector [10], an LM is a pseudo-discrete time series model. Typically,
LMs learn probability distributions over sequences of words, thus also facilitating the
generation of new text given some input, for example, in a language translation task.
While all LMs rely on neural networks, contemporary models almost exclusively
11
leverage the Transformer architecture [
]. Now, all of this begs the question—what
is the need for LMs in molecular discovery?
First, when applied to serializations of chemical entities (e.g., SMILES [12]), LMs
can learn highly structured representations, often even tailored for desired functional
properties [
13]. This allows performing a smooth and property-driven exploration
of the originally discrete protein or molecular space. A second attractive feature of
scientific LMs is their ability to seamlessly bridge natural and scientific languages.
This can give rise to ChatGPT-style chatbot interfaces that allow chemists to formulate their design objectives through natural language and to iteratively refine their
result with an interactive agent, thus potentially accomplishing complex chemical
tasks more rapidly. However, large-language models (LLMs) like GPT, which power
conversational agents, lack knowledge about scientific operations (e.g., molecular
discovery), access to information sources providing up-to-date data, and the ability
to accurately reference. They tend to hallucinate in their responses, which raises
questions about credibility, trust, and applicability. This critical challenge in AI for
scientific discovery can be overcome by integrating task-specific agents into LLMpowered conversational applications. The LLM can then reason over the appropriate
usage of these agents based on provided instructions. This approach eliminates the
application barriers associated with expert-developed AI models, which typically
require advanced programming and AI/ML skills from the intended user group, often
Соседние файлы в папке Библиотека им академика М.И. Перельмана