Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_5338_Библиотеки_им_академика_М_И_Перельмана
.pdf
102 M. Yamada and M. Sugiyama
https://t.me/med1917
11. Weisfeiler B, Lehman A (1968) A reduction of a graph to a canonical form and an algebra
arising during this reduction. Nauchno-Technicheskaya Informatsia 2(9):12–16
12. Rogers D, Hahn M (2010) Extended-connectivity fingerprints. Journal of Chemical Information
and Modeling 50(5):742–754,
13. Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In:
Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005, vol 2, pp
729–734,
14. Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2009) The graph neural network
model. IEEE Transactions on Neural Networks 20(1):61–80,
2008.2005605
15. Zhang Z, Cui P, Zhu W (2018) Deep learning on graphs: a survey. https://doi.org/10.48550/
ARXIV.1812.04202, https://arxiv.org/abs/1812.04202
16. Li Y, Tarlow D, Brockschmidt M, Zemel R (2015) Gated graph sequence neural networkhttps://
doi.org/10.48550/ARXIV.1511.05493, https://arxiv.org/abs/1511.05493
17. Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for
quantum chemistry. In: Proceedings of the 34th International Conference on Machine Learning
- Volume 70, JMLR.org, ICML’17, pp 1263–1272
18. Duvenaud DK, Maclaurin D, Iparraguirre J, Bombarell R, Hirzel T, Aspuru-Guzik A, Adams
RP (2015) Convolutional networks on graphs for learning molecular fingerprints. In: Cortes C,
Lawrence N, Lee D, Sugiyama M, Garnett R (eds) Advances in Neural Information Processing
Systems, Curran Associates, Inc., vol 28,
e311e65d81a9ad8150a60844bb94c-Paper.pdf
19. Battaglia PW, Hamrick JB, Bapst V, Sanchez-Gonzalez A, Zambaldi V, Malinowski M,
Tacchetti A, Raposo D, Santoro A, Faulkner R, Gulcehre C, Song F, Ballard A, Gilmer J,
Dahl G, Vaswani A, Allen K, Nash C, Langston V, Dyer C, Heess N, Wierstra D, Kohli P,
Botvinick M, Vinyals O, Li Y, Pascanu R (2018) Relational inductive biases, deep learning,
and graph networks.
20. Ernst D, Geurts P, Wehenkel L (2005) Tree-based batch mode reinforcement l earning. J Mach
Learn Res 6:503-556
21. Riedmiller M (2005) Neural fitted q iteration - first experiences with a data efficient neural
reinforcement learning method. In: Proceedings of the 16th European Conference on Machine
Learning, Springer-Verlag, Berlin, Heidelberg, ECML’05, pp 317–328
22. Kumar A, Zhou A, Tucker G, Levine S (2020) Conservative q-learning for offline reinforcement learning. In: Larochelle H, Ranzato M, Hadsell R, Balcan M, Lin H (eds)
Advances in Neural Information Processing Systems, Curran Associates, Inc., vol 33, pp
1179–1191,
1116a5085a6052-Paper.pdf
23. Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller M
(2013) Playing atari with deep reinforcement l earning.
5602, 1312.5602
24. Zhou Z, Kearnes S, Li L, Zare RN, Riley P (2019) Optimization of molecules via deep reinforcement learning. Scientific Reports 9(1):10752,
48-x
25. Schulman J, Moritz P, Levine S, Jordan M, Abbeel P (2015a) High-dimensional continuous
control using generalized advantage estimation.
26. Schulman J, Levine S, Moritz P, Jordan MI, Abbeel P (2015b) Trust region policy optimization.
https://doi.org/10.48550/ARXIV.1502.05477
27. Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization
algorithms.
28. Kakade S, Langford J (2002) Approximately optimal approximate reinforcement learning.
In: Proceedings of the Nineteenth International Conference on Machine Learning, Morgan
Kaufmann Publishers Inc., San Francisco, CA, USA, ICML ’02, pp 267–274,
eecs.berkeley.edu/~pabbeel/cs287-fa09/readings/KakadeLangford-icml2002.pdf
https://doi.org/10.1109/IJCNN.2005.1555942
https://doi.org/10.48550/ARXIV.1806.01261
https://proceedings.neurips.cc/paper_files/paper/2020/file/0d2b2061826a5df322
https://doi.org/0.48550/ARXIV.1707.06347
https://doi.org/10.1021/ci100050t
https://doi.org/10.1109/TNN.
https://proceedings.neurips.cc/paper/2015/file/f9b
https://doi.org/10.48550/arXiv.1312.
https://doi.org/10.1038/s41598-019-471
https://doi.org/10.48550/ARXIV.1506.02438
https://people.
Chapter 6
https://t.me/med1917
Subgraph-Based Molecular Graph
Generation
Masatsugu Yamada and Mahito Sugiyama
6.1 Introduction
De novo molecular design is an automated process for discovering molecules with
desired properties through computation. Recent developments of deep generative
models and reinforcement learning enable us to generate novel molecules for drugs.
However, de novo molecular design is still challenging in practice due to the massive
number of potential drug-like molecules, which is estimated to be between
60
10
[1]. Fundamentally, discovering molecules with desired properties is closely
related to the combinatorial optimization problem. In this chapter, we give recent
topics for molecular graph generation to overcome this challenge. We also introduce a graph mining-based method for generating molecules through “subgraph
structures” [
2] and state the future outlook on molecular generation.
Molecules are basically represented as graphs with node and edge attributes.
Representing chemical compounds as graphs is helpful to compute molecular
descriptors for predicting biological activities and chemical properties. Machine
learning models that utilize these descriptors are known as the Quantitative structureactivity/property relationship (QSAR/QSPR) models, which are often used for highthroughput screening of molecules satisfying required constraints and for searching
similar molecules. For this task, various software that can compute molecular descriptors from graph structures are released commercially, or freely. Recently, in machine
learning, various graph neural networks have been developed and they are also often
employed for QSAR.
Another way to represent the chemical structure is embedding it into a string,
which is called simplified molecular-input line-entry system (SMILES) [
3
23
10
]. SMILES
to
M. Yamada (B) · M. Sugiyama
National Institute of Informatics, Chiyoda-ku, Tokyo 101-8430, Japan
e-mail:
masatsugu-yamada@nii.ac.jp
M. Sugiyama
e-mail:
mahito@nii.ac.jp
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024
H. Satoh et al. (eds.), Drug Development Supported by Informatics,
https://doi.org/10.1007/978-981-97-4828-0_6
103
104 M. Yamada and M. Sugiyama
https://t.me/med1917
Fig. 6.1 A graph representation and its SMILES of aspirin. Left: A molecular graph and its
SMILES. Right: The lookup table for SMILES. The vertical axis shows the characters in SMILES,
and the horizontal axis shows the position of SMILES
corresponds to a depth-first tree traversal of a chemical graph, while it can be also
viewed as a sentence in natural language. The SMILES notation is helpful to denote
human-readable molecules and easily described in a simple rule. The structure of
chemical compounds can be denoted by the atom type (C, N, O, F, ...), the bond
type (=,
≡), or the branched symbols ((, )). For example, aspirin is denoted as
“O=C(C)Oc1ccccc1C(=O)O”. This string can be easily transformed into the
molecular graph structure. When considering SMILES as a sentence, SMILES can
be converted into one-hot table of indices and characters of SMILES as shown in
6.1.
Fig.
6.2 Related Work
In molecular generation, there are mainly two approaches: SMILES-based and graphbased approaches. To generate SMILES, one of the straightforward ways is to use
a deep generative model with recurrent neural networks, such as Long Short-Term
Memory (LSTM) [
et al. applied variational autoencoder (VAE) with GRUs and 1D CNN to generate
6
SMILES [
]. This character-based SMILES generation method can generate one-hot
tables composed of characters for representing SMILES and they can be converted
into molecules. In order to generate molecules with specific chemical properties and
high similarity for a target molecule, the latent vectors of encoder-decoder networks
are searched through Bayesian optimization, and the latent vectors are decoded into
] and Gate Recurrent Units (GRUs) [
4
5]. Gómez-Bombarelli
6 Subgraph-Based Molecular Graph Generation 105
https://t.me/med1917
one-hot tables, followed by converting them into target molecules. However, reconstructing SMILES from one-hot tables is often highly challenging because representation patterns of SMILES are difficult to learn without considering SMILES
grammar rules. To address this problem, the VAE model that generates the SMILES
grammar rule instead of characters is proposed [
Yang et al. [8] proposed an algorithm, called ChemTS, that combines a pretrained
RNN-based SMILES generative model with Monte Carlo Tree search (MCTS) to
optimize the order of each character in SMILES and their properties simultaneously. ChemTS can generate target molecules with desired properties by searching
an appropriate character through MCTS. Yang et al. [
into the massive parallel computation of MCTS to generate and search molecules.
Olivecrona et al. [
an RNN-based generative model and reinforcement learning algorithm, which is
a Monte Carlo variant of policy gradient (REINFORCE). In reinforcement learning,
the constraints, such as excluding some specific structure, can be incorporated as a
reward function and the policy network for generating various target molecules is
fine-tuned. These algorithms have the same system in common that each character
in SMILES is sequentially generated step-by-step.
Instead of SMILES-based strategies, You et al. [11] proposed a graph-based
method called GCPN, which is a node-wise graph generation model while satisfying
with valid graph structures with guiding specific properties through reinforcement
learning. The state of an intermediate graph is represented as node embeddings by
a graph convolutional network, and a policy network takes four actions to construct
a graph. The policy samples two node ids (atoms), the edge type, and the terminal
condition from the current state. The policy network is optimized through the Proximal Policy Optimization. These methods can generate valid molecules with desired
properties at the final step. Shi et al. [
for molecular graph generation, called GraphAF. In GraphAF, parallel training can
be performed unlike GCPN when generating molecules through masks between an
input molecule graph and an output latent variable.
Jin et al. [13
Nodes in a junction tree represent subgraphs extracted from a molecular dataset,
and a graph neural network determines which nodes or edges are combined with
each other in the junction tree. To search molecules that optimize the desirable properties, it is necessary to search two vectors, what a tree structured scaffold is and
how a molecule is reconstructed within latent embedding space. Takeda et al. [
proposed to generate molecules by combining substructures that contribute to the
target properties, where candidate molecules are searched by McKay’s Canonical
Construction Path (MC-MCCP) algorithm [
multi-objective molecule generation using interpretable substructures as rational
for extracting substructures by MCTS to generate molecules by merging common
substructures and graph completion.
Yamada and Sugiyama [2] proposed a graph-based method called MOLDR, which
combines decomposition of molecules into subgraphs by graph mining and reassembling of subgraphs to generate new molecular graph. A related approach exists in
10] proposed an algorithm called REINVENT, which utilizes
] proposed a Flow-based autoregressive model
12
] proposed a VAE model that generates junction trees over molecules.
7].
9] also extended their ChemTS
15, 16
]. Jin et al. [17] also proposed
14]
106 M. Yamada and M. Sugiyama
https://t.me/med1917
the task of planning of chemical synthesis, which also combines subgraphs and
MCTS [
18], while it is not applied to the property optimization. Geng et al. [19]
proposed the framework called MiCaM, which also leverages a decomposition-andreassembling framework to generate new molecules. MiCaM is a merge-and-update
algorithm to mine the most frequent motifs and to build a motif vocabulary. In
the reassembling step, a connection-aware decoder to reassemble those motifs is
proposed for molecular generation.
6.3 The MOLDR Algorithm
In this section, we introduce our method MODLR [2] that mines subgraph structures
and generates new molecules with desired properties via reinforcement learning
instead of using deep generative models. Although deep generative models can
generate diverse molecules, searching latent space for generating molecules with
target properties is usually difficult and its generating process often black box. Our
framework is more effective in the situation where structural motif is already determined or is assumed to be within training dataset, rather than generating molecules
that chemists never imagined. Another advantage of our method is to show the
trajectory of generating process.
6.3.1 Graph Decomposition via Frequent Subgraph Mining
Given a collection of graphs as an input molecular dataset, our idea is to apply
frequent subgraph mining [
appear in the graph dataset. Formally, given a graph dataset
the objective of frequent subgraph mining is to find all subgraphs G satisfying the
condition
support(G) ≥ minsup, where support(G) is defined as
that is, the number of graphs in D that contains G as a subgraph, and minsup ∈ Z is
a frequency threshold.
We use the gSpan [21] algorithm, which is commonly used for the task of frequent
subgraph mining. It enumerates subgraphs in a depth-first manner. Each graph is
represented as the so-called DFS code, which is constructed from a search tree based
on a lexicographic order and enables us to efficiently check duplication of enumerated
graphs as shown in Fig.
enumeration, it checks whether or not its DFS code is canonical, meaning that the
order of DFS code is in its minimum order. After completion of gSpan, we check
every enumerated subgraph and keep only subgraphs whose target property score is
20
] to the dataset, which finds subgraphs that frequently
D ={G1, G2, ..., Gn},
|
|
}
, (6.1)
|
i
support(G) =
|
|
{
G
∈ D | G ⊑ G
|
i
6.2. More precisely, for each explored graph during the
6 Subgraph-Based Molecular Graph Generation 107
https://t.me/med1917
already higher than some threshold, which is determined beforehand, to efficiently
reassemble them to construct new graphs in the next reassembling step.
Molecules are firstly converted into molecular graphs, where each node represents
an atom type and each edge represents a bond type. However, if we directly apply
gSpan to such molecular graphs, it gives a lot of invalid subgraphs in terms of
molecules as building blocks for molecular generation as shown in Fig.
6.3.Thisis
because gSpan does not know the chemical context and simply enumerates frequent
subgraphs, hence, for example, the ring structure will be truncated by gSpan, while
such truncated subgraphs are invalid and unnecessary for the reassembling step.
To circumvent this problem, we apply tree decomposition to molecular graphs as
preprocessing before applying gSpan, and convert them into molecular junction trees.
Fig. 6.2 An example of gSpan applied to two molecular graphs under support two. In this condition,
all subgraphs are enumerated in a lexicographical order
Fig. 6.3 An example of directly applying gSpan to molecular graphs. The enumerated subgraphs
include truncated structures of molecules. Chemical properties such as ring of benzene are
completely ignored and more graphs are mined through gSpan
108 M. Yamada and M. Sugiyama
https://t.me/med1917
A tree decomposition maps a graph G = (V , E) into a junction tree T = (V, E ),
V ={C1,..., Cn} is a collection of subsets of V ; that is, each Ci ⊆ V , and
where
E is a set of edges between elements of V. A junction tree satisfies the following
properties:
1. The union of all sets C1, ... , Cn equals to V; that is,
2.
For every edge (u, v) ∈ E, there exists Ci ∈ V such that u ∈ Ci and v ∈ Ci.
U
i
Ci = V .
3. If Ck is on a path from Ci to Cj in T , Vi ∩ Vj ⊆ Vk.
By converting a graph into its corresponding junction tree, by definition, each
cycle will be gathered as a single node and all cycles will be eliminated. Therefore,
if we apply gSpan to not the original graphs but the converted junction trees, we
can avoid enumerating invalid subgraphs in which the ring structure of a molecule,
represented as a cycle on a graph, is truncated. In addition, gSpan on junction trees
can dramatically reduce the number of frequent subgraphs as shown in Table
6.1 shows results of applying gSpan to the ZINC database with varying the
Table
6.1.
minimum support. We can see that enumeration based on junction trees is much
faster than directly applying gSpan to molecular graphs. This result means that our
junction tree-based enumeration is effective in the real-world ZINC database. When
the minimum support is 100, we were able to find a large number of subgraphs, and
it is expected that we have collected enough amount of substructures. This is also an
advantage of using junction trees in the decomposition step for molecular generation.
The edge label information and the node label information in each clique are lost
in a junction tree, hence we need to restore them after frequent subgraph mining.
To achieve this task, we use a subgraph matching algorithm that matches between
original graphs and obtained trees. We use the indexed-based subgraph matching
algorithm with general symmetries (ISMAGS) [
is usually not so large and the number of nodes is mostly around 20
22]. Since the size of each molecule
∼30 in the task
of molecular generation, this restoring process is not computationally expensive.
6.4 shows examples of building blocks of substructures extracted from ZINC
Figure
250k filtered by the score of
QED > 0.7 with minimum support of 100. These
structures can be used as building blocks for molecular graph reassembling.
6.3.2 Graph Reassembling from Frequent Subgraphs
In our reassembling approach, we focus on the adjacency matrix of frequent
subgraphs obtained by the previous graph decomposition step. The operation of
reassembling two graphs is equivalent to concatenating two adjacency matrices into
one matrix and deleting some indexes that are merged. The main difference of our
approach to using subgraphs as building blocks and the existing approaches is that
we do not use node-wise graph generation or step-by-step character generation. In
the text generation approach [
ular character which denotes chemical state, such as the atom (C, N, O, F, ...), the
bond type (=,
≡), or the branched symbols, from the set of character types occurred
23] based on SMILES, an algorithm picks up a partic-
6 Subgraph-Based Molecular Graph Generation 109
https://t.me/med1917
Fig. 6.4 Examples of extracted substructures sorted by the score of QED. ZINC 250k molecules
are decomposed into junction trees, gSpan enumerates frequent subtrees, and are reconstructed into
molecules by ISMAGS. These substructures become building blocks for molecular reassembling
Table 6.1 Comparison of frequent subgraph enumeration with or without junction trees
Minimum support Number of mined trees Number of mined graphs
100,000 0 23 (1334 sec)
10,000 8 (164.42 sec) 4040 (106.5 min)
5000 39 (216.21 sec) –
1000 910 (342.20 sec) –
100 23,616 (775.23 sec) –
“–” means that computation did not stop in 2 h
in a training dataset to generate and expand molecules. Likewise in the node-wise
graph generation [
edge type between source and target atoms from the candidate set of atom and edge
types. Our method can be more powerful in terms of intermediate molecules as we
directly combine subgraphs that already have desirable properties as building blocks
in a generation step.
In order to assemble molecular subgraphs, we pick up two graphs Gt and G
building blocks and combine them to generate a new graph
number of building up steps of molecules. Let us assume that
with V (Gt) ={v1,..., vn} and G
In the reassembling with nodes, we select single nodes
such that they have the same node labels: lv(vi) = lv(uj). We overlay these two nodes
v
as
V (G
Gt and G
replaced with
for converting the state of a hypergraph into a graph; that is, V (G
t+1
) ∪ V (G
t
'
) \{vi, uj}∪{v
t
'
are preserved in G
t
(v
11, 24], an algorithm selects a node type (atom symbol) and the
G
, where t is the
t+1
Gt = (V (Gt), E(Gt))
'
= (V (G
t
'
), E(G
t
'
)) with V (G
t
'
) ={u1,..., u
t
vi ∈ V (Gt) and uj ∈ V (G
t+1
, vk ) or (v
t+1
} for a newly constructed graph G
t+1
, where if there is an edge (vi, vk ) or (uj, ul),itis
t+1
, ul).
t+1
. All edges in
t+1
'
from
t
'
}.
n
'
t
) =
)
110 M. Yamada and M. Sugiyama
https://t.me/med1917
Fig. 6.5 Procedure of merging node between two graphs. Left figure shows the two graphs represented by one graph. Red rectangle encircling
figure shows the result of reassembling two graphs. After merging nodes, the bits written in red
number are added into adjacency matrix and the index of merged node
column
(∗, ∗) is merged into one row and column. Right
∗ is removed from row and
Fig. 6.6 Procedure of merging an edge between two graphs. Left figure shows the two graphs
represented by one graph. Red rectangle encircling
encircling
the bits written in red number are added into adjacency matrix. Selecting any two nodes is available,
but the possibility of producing invalid molecules increases
C0, C1. Right figure shows the result of reassembling two graphs. After merging edges,
C6, C7is merged from dotted reg rectangle
6 Subgraph-Based Molecular Graph Generation 111
https://t.me/med1917
Fig. 6.7 An example of merging process. Selecting two nodes between graphs results in an invalid
molecule
In the reassembling with edges, we select edges from rings, and overlay them
in the same manner as the assembling with edges. This assembling is similar to
reconstructing a graph from a junction tree; that is, nodes of a clique in a junction
tree have intersected nodes that are connected with each other between subgraphs.
Assembling two graphs is equivalent to choose the intersection of nodes or edges.
The concrete procedure of merging node and edge is shown in Figs. 6.5 and 6.6.
In order to combine two graphs, two adjacency matrices should be concatenated.
The process of merging nodes and edges is operated through the adjacency matrix.
6.5, two graphs are merged with nodes by adding the bit in adjacency matrix
In Fig.
and by removing duplicated nodes. Figure
merged into one graph with two rings.
6.6 shows that two graphs G1 and G2 are
C6 and C7 is overwritten into C0 and C1 in
this case so that a bit in the adjacency matrix is added at the position that is merged.
Unneeded nodes are removed in this process. In the example of merging edges, we
chose the adjacent nodes, but any nodes can be selected. In case of selecting two
nodes that are not adjacent, the resultant may include the huge ring structure or be
an invalid molecule as shown in Fig.
6.7.
Figure 6.8 shows the process of reassembling the two molecular graphs with
nodes or edges, respectively. The candidate set of node label C for merging is
{v0 :{u6, u7, u9, u10, u15}, v4 :{u6, u7, u9, u10, u15}, v5 :{u6, u7, u9, u10, u15}},
where indices of nodes correspond to numbers in the illustration in Fig.
u
do not include internal nodes such as
as the resulting graph will be an invalid
12
6.8.We
molecule. In the reassembling with edges, the candidate set in the same node label
(C, C) is
{{(v0, v1), (u6, u7)}, {(v1, v2), (u6, u7)}}, resulting in two new graphs.
The computational cost of combining two graphs depends on the number of nodes
and the number of edges in rings. In the worst case, where we need to consider all
combinations of nodes and edge of two graphs
complexity becomes
O(|V ||V'|+|E||E'|). However, this can be usually reduced
G = (V , E) and G'= (V', E'),the
in practice by considering the symmetrical structure of a graph and some type of
restrictions of chemical valency of an element in the case of molecules. We always
check these conditions whenever a new molecule is generated and remove it if it does
not satisfy such conditions. Therefore, the molecules generated by our method are
always valid.
Соседние файлы в папке Библиотека им академика М.И. Перельмана