Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_5338_Библиотеки_им_академика_М_И_Перельмана
.pdf
62 R. Yoshida
https://t.me/med1917
with statistical surrogate models. By contrast, the inverse problem makes predictions in the opposite direction. Specifically, by setting a target value for output
and finding the inverse mapping of the forward predictive model, the input X that
achieves the desired output is determined in an exact or approximate manner. These
calculations involve the “representation,” “learning,” and “generation” of the material. Typically, a descriptor vector is used to “represent’” the input
X to handle the
non-numeric object, such as a molecule, compositional input, or crystalline system,
into a machine learning framework, and the mathematical mapping from
X to Y
is “learned” from a given dataset using a supervised learning technique. Inverse
mapping is then used to “generate”
X with the desired Y to identify promising
design candidates.
This chapter describes the basic concept and key technologies of machine learning
for forward and inverse analyses of material design, illustrated with practical applications from our recent works, for example, on soft matter [
7, 8]. In particular, we describe our strategy and attempt to build a material data
[
platform to overcome the significant challenge of limited data [
6] and quasicrystal research
9]. The most impor-
tant resource in data-driven materials research is data. However, data i n materials
research is extremely scarce. Moreover, data do not exist in the neighboring area
of innovative materials. Therefore, the interpolative prediction of a conventional
data-driven method does not lead to the discovery of truly innovative materials. To
overcome this barrier, machine-learning systems that integrate and analyze limited
experimental datasets and systematically generate data from computer simulations
play an important role.
Y
Fig. 4.1 General workflow of machine learning in materials informatics (MI), consisting of forward
and backward prediction tasks. The input variable is converted to a descriptor vector, and a forward
model is trained to predict the output variable from a given vectorized input in a forward direction.
The inverse mapping of the forward model is then solved to identify the input variables that exhibit
the desired output
4 Materials Informatics with Limited Data 63
https://t.me/med1917
4.2 Machine Learning Applications in Materials Science
4.2.1 Bayesian Molecular Design: Discovery of Thermally
Conductive Polymers
Ikebata et al. (2017) [5] developed a machine-learning algorithm for molecular design
based on Bayesian inference. The idea of using the Bayesian inference for molecular
design tasks was originally presented by Funatsu and co-workers in 2010 [
4]. The proposed method derives an inverse prediction scheme from a forward
2016 [
predictive model based on Bayes’ rule of conditional probabilities.
p(X |Y ∈ U ) ∝ p(Y ∈ U |X )p(X ). (4.1)
Using the forward model, which predicts material property Y from a given
molecule
this model, the probability that a given
calculated as
the context of Bayesian inference,
parameter
probability distribution
X , the conditional probability distribution p(Y |X ) is determined. From
X falls into the desired property range U is
p(y|X )dy, as shown on the right side of Eq. (4.1). In
p(Y ∈ U |X ) =
U
p(Y ∈ U |X ) is called the likelihood function of the
X . Furthermore, the promising search space is narrowed down via the prior
p(X ). The conditional probability distribution p(X |Y ∈ U )
on the left-hand side is called the posterior probability distribution. The posterior
probability distribution is proportional to the product of the likelihood function and
the prior probability distribution. From this conditional probability distribution,
is sampled to identify new molecules that satisfy the desired properties. The idea
of utilizing t he Bayesian inversion formulas for molecular design was originally
].
presented in [
3, 4
Ikebata et al. (2017) [5] developed a molecular structure generator based on
a probabilistic language model called an extended n-gram. The chemical struc-
X of an existing compound used in the training dataset is described by the
ture
SMILES representation [
16
], where
X is represented by a string of length p as
X = x1x2 ... xp. Using this string set of molecules synthesized thus far, an n-gram
language model was trained to construct a structure generator that mimicked the
patterns (frequent fragments, appropriate chemical bonding rules, etc.) that appear
in existing molecules. The probability distribution
p(X ) of string X is expressed as
a product of the conditional probabilities as follows:
3] and
X
The occurrence probability of the i-th letter xi depends on the preceding
x
,..., x1. Generally, SMILES representations of the same chemical structure are
i−1
not unique. These structurally equivalent strings are treated as different. The basic
p(X ) ∝ p(x1)
p
i=2
p(xi|x
,..., x1,). (4.2)
i−1
64 R. Yoshida
https://t.me/med1917
concept of structure generation based on the language model is as follows: The conditional probability
p(xi|x
,..., x1,) is estimated from the frequency of substrings of
i−1
known compounds, and the model is trained to learn the chemical language context.
x
... x
For a given substructure
1,
, the remaining strings are generated using the
i−1
model. Characters are added sequentially according to conditional probability until
the end code appears. The language model must produce strings that match SMILES
grammar rules. Ikebata et al. (2017) [
a specific type of modeling for the conditional distribution
5] overcame this technical difficulty by using
p(xi|x
,..., x1,) and
i−1
modifying the word definitions of the SMILES strings.
Wu et al. (2019) [6] applied a Bayesian molecular design algorithm to identify
polymers with high thermal conductivity. The workflow of the data analysis is illustrated in Fig.
sufficiently high softening (high glass transition and melting temperatures T
respectively) but do not soften or melt up to high temperatures. Specifically, the
T
m,
4.2. In general, polymeric materials with high thermal conductivity have
and
g
more rigid the polymer, the less conformational change it can undergo after melting,
the lower its entropy of melting, and the higher its melting point. The glass transition temperature of a polymer is governed by intermolecular forces, bending, and
symmetry. Polymeric materials with main chains having a high proportion of ring
structures have a higher T
owing to the intermolecular interactions and cohesive
g
forces involved in the heat of melting. Therefore, aromatic polyamides with high T
and Tm values were targeted in the molecular design.
To obtain the training dataset, 5917 and 3234 homopolymers with experimentally
observed T
erties database PoLyInfo [
and Tm values, respectively, were extracted from the polymer prop-
g
10], and a forward model was constructed using these
data. Molecular fingerprints, which quantify the pattern of the molecular skeleton
g
Forward prediction: polymeric properties
Polymer (S)
Backward prediction: designing polymers
Library generation
Fig. 4.2 Machine learning workflow of designing polymers with high thermal conductivity
Descriptor (φ)
01000100000011010011
with desired properties
Synthesis of new polymers & thermophysical property measurements
Thermal conductivity 80% higher than that of
typical polyimides
High thermostability
Soluble in organic solvent
Ease of film formation
Properties (Y)
λ Thermal conductivity
Glass transition temper.
T
g
Melting point
T
m
C
Heat capacity
p
Pre-trained
model for proxy
properties
Prediction model
Tra nsfe r
learning
Prediction of thermal conductivity
of thermal
conductivity
4 Materials Informatics with Limited Data 65
https://t.me/med1917
and the local structural features of an input monomer, were used as input descriptors. For a given set
F of substructures, the chemical structure is encoded into the
descriptor vector, with each element indicating the presence or absence (binary type)
and frequency (count type) of a substructure
descriptor vector takes the value 1 if
Count-type descriptors have the occurrence number of
fi ∈ F . Element i of the binary-type
X has the substructure fi and 0 otherwise.
fi as an element.
We solved the inverse problem of the forward model and created a virtual library
of 1000 virtual polymers with the predicted T
and Tm values in the range of 200–
g
500 °C and 300–600 °C, respectively. However, the heat resistance in the design
of thermoplastic resins that can be melt-molded must be reduced slightly. For this
reason, an upper limit of 300 °C was set for the T
in the post-screening stage.
g
Next, the thermal conductivities of the candidate molecules were estimated using
a machine-learning model. Because PoLyInfo had only 28 samples with amorphous
structures and an observed thermal conductivity at approximately room temperature, a satisfactorily well-performing prediction model could not be constructed
using conventional supervised learning. To overcome this limitation, we introduced
a technique known as transfer learning [
11–15]. Transfer learning is an analytical
technique in which a model previously trained on a source task is transferred to
a target task. By appropriately setting up source tasks and models for pretraining,
models with high prediction accuracy can be constructed, even with a small amount
of data, using transfer learning. Data, such as the glass transition temperatures of the
polymers and specific heat capacities, were used for pretraining, as detailed later.
The procedure is described in detail in Sect.
4.3.3.
Based on the thermal conductivities predicted by the transferred model, the
candidates were further narrowed down to three aromatic polyamides, and polymer
synthesis and physical property measurements were performed. One of the synthesized polymers was found to have a thermal conductivity of 0.41 W/(m・K).
This corresponds to a performance improvement of approximately 80% compared
with that of typical unoriented polyamides. In addition, the synthesized polymers
possessed various properties required for practical use, such as high heat resistance,
solubility in organic solvents, and ease of film processing.
4.2.2 Synthetic Route Design
After designing the chemical structure, we designed a synthetic pathway for a
candidate molecule. For example, consider the following two-step synthetic reaction:
In the first step, the two reactants X1 and X2 react to synthesize the intermediate
product
Z. This was then fed with reactant X3 to synthesize the final product Y .
The problem is to find a pair of reactants,
target molecule, Y = Y ∗. The reactants were selected from commercially available
X1 + X2 → Z + X3 → Y . (4.3)
X = {X1, X2, X
}
, that can reach the
3
66 R. Yoshida
https://t.me/med1917
compounds. Typically, O10
design. Therefore, the problem boils down to combinatorial optimization, where the
search space
T consists of O10
6
commercial compounds are used in the synthetic route
6×3
candidate pathways.
The forward problem aims to derive a predictive model Y = f (X ) for product Y
from a set X of reactants. In the inverse problem, on the other hand, given a product
−1
Y = Y ∗, the inverse map X = f
target
∗
is obtained. Recent developments in
Y
(
)
deep learning have significantly improved the accuracy of the forward prediction of
synthetic reactions. We addressed the forward prediction task based on the SMILES
representation [
16]. As shown in Fig. 4.3, the two reactants X = {X1, X
}
of a one-
2
step synthetic reaction are converted into a string based on SMILES notation, and
both are connected by a period. The chemical structure
Y of the product was also
converted to a SMILES string. Thus, the prediction of a one-step synthetic reaction
can be formulated as a string-to-string-mapping problem. The model uses a deep
learning technique for machine translation. For example, the USPTO database of
synthetic reactions of U.S. patented compounds contains approximately one million
data points [
the transformer [
19].
[
17]. Using this dataset, we trained a machine translation model called
18], which has been shown to achieve over 90% prediction accuracy
Guo et al. (2020) [20] proposed an algorithm that finds the inverse map of the
forward model of synthetic reaction prediction and searches for a pair of r eactants
that synthesize an arbitrarily given product. The posterior distribution
p(X |Y = Y
∗
is modeled as follows:
)
Fig. 4.3 Synthetic route design based on the forward and backward predictions using a machine
translation model based on the SMILES representation of synthetic reactions. This figure is a reprint
from Guo et al. (2020) [
20]
4 Materials Informatics with Limited Data 67
https://t.me/med1917
pX |Y = Y
∗
∝ pX , Y = Y
1
∗
=
exp−
Z
E(Y
∗
, f (X
T
))
(4.4)
The energy E of the Gibbs distribution represents the dissimilarity between the
fingerprint descriptor of the target product and the predicted product of the forward
model. The temperature parameter
T is a hyperparameter that controls the diversity
of candidate reactants. The posterior distribution was defined for all combinations
of the commercial compounds. For example, in the design of the two-reactor, t wostep reaction route described in Eq. (
4.3), the domain consists of O10
6×3
discrete
points. Therefore, the posterior distribution is expressed as:
pX |Y = Y
∗
∝
pXi, Y = Y
Xi∈T
∗
I(X = X
. (4.5)
)
i
The indicator function I (.) takes 1 if the argument is true and 0 otherwise. In
other words, the posterior distribution is a discrete distribution with probability
p(Xi|Y = Y
∗
)
∝ p(X
, Y = Y
i
Xi ∈ T . Since this probability distribution cannot be calculated exactly, n repre-
sentative candidate points
∗
X=X
over a substantial number of candidate points
)
|i = 1, ··· , nare selected and approximated as
i
follows:
n
ˆ
pX |Y = Y
∗
∝
i=1
ˆ
p
Xi, Y = Y
∗
I
X = ˆX
. (4.6)
i
The set of candidate points used for approximation should be as large as possible
p(Xi|Y = Y
∗
)
∝ p(X
, Y = Y
i
∗
and contain a variety of reaction paths.
)
Guo et al. (2020) [20] developed a sequential Monte Carlo algorithm to derive
an approximate distribution. Figure
4.4 shows an example of the predicted two-step
reaction route for the target molecule. In this example, more than 6000 reaction
routes were predicted and classified into clusters based on their chemical structure
patterns. The synthetic routes representing each cluster were selected and plotted,
and the synthetic feasibility of the candidate routes was evaluated based on the
knowledge of organic synthesis (shown as a score in Fig.
4.4). Such evaluation tests
were conducted for several target molecules, and it was concluded that 35–50% of
the candidate pathways were chemically plausible. The results of the data analysis
are presented as hypotheses to stimulate the creativity of the user, who is an expert
chemist, and support decision-making. These cases require diverse hypotheses and
scenarios. If the final decision is left to the experts, the hypotheses can contain errors.
Even at the expense of the accuracy of the hypotheses, a variety of scenarios should
be presented to induce novel ideas that cannot be reached by the experience and
knowledge of experts alone.
Zhang et al. (2023) [21
] formulated the problem of simultaneously designing
molecules with the desired set of properties and synthetic routes within the Bayesian
inference framework. This task was treated as an inverse problem, in which a forward
68 R. Yoshida
https://t.me/med1917
Fig. 4.4 Synthetic route design by machine learning. Ten candidate routes for the target product
are shown. Each step of the reaction was evaluated by an expert in organic synthesis (1: possible,
2: uncertain, 3: not possible). This figure is a reprint from Guo et al. (2020) [
model was defined as a cascade of a reaction prediction model and property predictors. The overall search space for the reactant sets spanned all possible combinations
of commercially available compounds. The structure of the reaction network is also a
design variable that involves the number of reaction steps and network topology. For
example, the pattern of branching routes, width and depth of networks, and number
of leaf nodes were included in the design variables. To solve this difficult combinatorial problem, we presented a powerful sequential Monte Carlo algorithm that
recursively designs a synthetic reaction network by sequentially building single-step
reactions.
20]
4 Materials Informatics with Limited Data 69
https://t.me/med1917
4.2.3 Machine Learning to Discover New Quasicrystals
Quasicrystals do not have the translational symmetry of ordinary crystals but have a
high degree of order in their atomic arrangement. The first quasicrystal was discovered in 1984 by Dr. Dan Shechtman (Nobel Laureate in Chemistry, 2011) [
then, approximately 100 stable quasicrystals have been identified [
38]. In the history
of quasicrystal research, the discovery of new quasicrystals has led to the discovery
of new physical phenomena, such as anomalous electronic properties, insulating
behavior, valence fluctuations, quantum criticality, and superconductivity. However,
in recent years, the pace of the discovery of new quasicrystals has slowed significantly. This trend is primarily due to the lack of clear design guidelines for the
synthesis of new stable quasicrystals.
Liu et al. (2021) [7] and Liu et al. (2023) [8] formulated the problem of predicting
quasicrystals using a simple workflow of supervised learning (Fig.
variable of the model is chemical composition. The output variable represents a class
label indicating “quasicrystals” (QCs), “approximant crystals” (ACs), and “others,”
including ordinary periodic crystals. ACs are periodic crystals composed of a local
structural unit, such as an icosahedral cluster, similar to the corresponding QCs.
Importantly, ACs are formed in nearby compositions of QCs, indicating that their
stabilization mechanisms are very similar.
As training data, we used the chemical compositions of the QCs, ACs, and ordinary crystals discovered to date. From previous literature, we compiled a list of 80
thermodynamically stable quasicrystals and 78 approximants found to date. We also
randomly extracted the chemical compositions of 10,000 periodic crystals from the
Materials Project database [
22]. In addition, 90 compositions that failed to synthe-
size quasicrystals in our laboratory experiments were added to the training data for
the class “others.” For each class, approximately 80% of the randomly selected data
were used to train the model (66, 60, and 8072 for QC, AC, and others, respectively), and the masked remainder were used as test data to evaluate the prediction
performance (14, 18, and 2018 for QC, AC, and others, respectively). To adjust the
hyperparameters, cross-validation was performed within the training dataset, and
the parameters that minimized the generalization error in the validation set were
selected. To account for the effect of sampling in the evaluation of the forecasting
performance, the training and test sets were randomly selected 100 times, and the
mean and variance values of the performance indices of the individual models were
calculated.
The input chemical composition is represented by X = X
1
c
1
element of the descriptor vector of length 232 takes the following form:
The ηX
ical element
Table S2 in [
1
ψ
(X )
g,η
k
on the right-hand side represents a feature defined for each chem-
= gc
,..., cK ,ηX
1
,...,ηX
X k, such as atomic number, electronegativity, and polarizability (see
8] for details). The function g is applied to the elemental features
K
37]. Since
4.5(a)). The input
2
c
... X
2
K
. Each
c
K
X
(4.7)
70 R. Yoshida
https://t.me/med1917
Fig. 4.5 Machine learning to predict quasicrystal-forming chemical compositions. This figure is a
reprint from Liu et al. (2023) [
(QC/AC/others) for any given chemical composition. b Predictive performance summarized with
a confusion matrix, precision, recall, and F
diagrams with the three aluminum alloy quasicrystals discovered. Electron diffraction patterns of
the three discovered quasicrystals (Al
8]. a Machine learning workflow to construct three-class discriminator
score with respect to test instances. c Predicted phase
1
Ni20Os15,Al78Ir17Mn5,and Al78Ir17Fe5)
65
ηX
1
,...,ηX
K
and composition c1,..., cK to calculate one element
g,η
(
of
)
the descriptor vector. The weighted mean, weighted variance, max pooling, and min
pooling are used as
XenonPy [
11, 23
g. The Python library for MI being developed by our group,
], implements 58 different element features [7]. The elemental
features include atomic number, bond radius, van der Waals radius, electronegativity,
thermal conductivity, band gap, polarizability, boiling point, and melting point. By
combining the 58 elemental features and the four types of summary statistics, each
input composition was characterized by a 232-dimensional (
58 × 4) descriptor.
The trained model was able to predict the binary classification task of quasicrys-
tals with more than 95% accuracy (Fig.
4.5(b)). Liu et al. (2023) [8
]usedthis
model to perform an exhaustive screening of 1,080 alloy systems, which correspond to the entire space of aluminum ternary alloys. Quasi-crystalline phases
were predicted to exist in 185 alloy systems. From these, we excluded those for
which the phase diagrams had been reported in previous studies and narrowed
the list to 30 candidates. For the first time, Al-Ni-Os, Al-Ir-Mn, and Al-Ir-Fe
were selected, and synthesis experiments revealed the presence of quasicrystalline
phases (Al
Fig.
65Ni20Os15
4.5(c), the phase diagrams for Al-Ni-Os, Al-Ir-Mn, and Al-Ir-Fe were success-
,Al78Ir17Mn5, and Al78Ir17Fe5) in all the systems. As shown in
fully predicted for the three discovered quasicrystals. All three quasicrystals were
4 Materials Informatics with Limited Data 71
https://t.me/med1917
observed after a long annealing process, suggesting that they were thermodynamically stable. In addition, the electron diffraction patterns obtained using a transmission electron microscope revealed that all three materials had decagonal quasicrystal
structures (Fig.
4.5(c)).
4.3 Machine Learning for Limited Data
Data on materials research are extremely scarce compared to data from other fields.
There are three possible reasons for this: (a) the high cost of data production and
accumulation; (b) the diversity of researchers’ needs, which makes it difficult to create
open data that can be shared with society; and (c) the high awareness of the need to
keep information confidential from competitors, which makes it difficult to create
incentives for data release. Against this background, the trend toward the creation
of open data through community cooperation is extremely weak. Therefore, in the
short to medium term, data that can be produced in university laboratories or by a
single company are expected to become the standard resource for machine learning
in materials research. In addition, there is a lack of data on innovative materials.
However, in unexplored areas where data is lacking, the “interpolative predictions”
of machine learning lose their validity. Therefore, in principle, a simple, data-driven
approach cannot lead to the discovery of innovative materials. In other words, the
essence of the problem in data-driven materials research is the lack of data.
4.3.1 Transfer Learning
In transfer learning, the target and source domains are considered. The target domain
is the system to be predicted. For example, consider a situation in which a certain
property in the target domain has to be predicted. However, experimental data are too
scarce to apply conventional machine learning. In addition to the data in the target
domain, the lack of data can be compensated for by integrating a large computational
database of physical properties in the public domain, datasets accumulated in past
research projects, and a pretrained model of a different property in the source domain.
We outline a supervised transfer learning method using neural networks (NNs).
An NN represents the mapping from the input to the output with a multilayered
composite function. NNs trained in the source domain represent general features
in layers close to the input and are transformed into task-specific features as they
approach the output.
First, a model Y
domain, where
◦ f
L−1
◦ ··· ◦ f
◦···◦f
f
L
f
K ◦fK−1
of the model for the target domain. That is, the model Y
= f
source
Y
source
(X )
1
up to layer K (K < L) of this learned model is used as a descriptor
(X )
1
source
and X denote the input and output variables, and f
represents an L-layer NN. The partial model φ
is constructed using a dataset in the source
(X )
target
= f
target
φ
(
source
source
source
(X )
(X )
(X ))
=
=
is
Соседние файлы в папке Библиотека им академика М.И. Перельмана