Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_5587_Библиотеки_им_академика_М_И_Перельмана.pdf
Importance of Applicability Domain of QSAR Models
ternative measures (as e.g. Euclidean distance and Pearson correlation coefficient) due to its higher
accuracy and practical requirements to store ensemble predictions for all training set molecules (Tetko
& Tanchuk, 2002).
3.11 Concordance of a Classification Ensemble
The highest number or the relevant percentage of the models that produce the same prediction can be
used as a measure of the concordance of an ensemble (Chen et al., 2005). As an example, if there are
5 models that give predictions {a; b; a; a; a}, then the concordance is 4 (or 80%). The measure that is
opposite to the concordance (i.e. a- CONCORDANCE) can be used as a distance to model. The idea
behind this method is similar to that of the standard deviation (STD) distance to model approach, but is
adapted for classification models and qualitative predictions.
4. PROBABILITY DENSITY DISTRIBUTION
• Theory: Probability density distribution is one of the most advanced approaches for defining AD,
based on estimating the probability density function for the given data. The approach is classified
into two classes. The first one is a parametric method which assumes a standard distribution such
as Gaussian and Poisson distributions, and the second one is a non-parametric method which does
not rely on such assumptions considering the data distribution (Jaworska et al., 2005; Netzeva et
al., 2005).
These methods are executed by estimating probability density of the compounds followed by identifying
Highest Density Region which consists of a known fraction (as set as user input) from the total probability mass (Netzeva et al., 2005). A potential is formed for each compound in the training set such that
it is highest for that molecule and decreases with increasing distance. Once the potential is evaluated for
all the compounds, global potential is obtained by summing up the individual potentials thus indicating
the probability density (Forina et al., 1991; Jouan-Rimbaud et al., 1999).
The most important feature of these methods is their capability to identify the internal empty space. The
actual data distribution can be revealed by generating concave regions around the interpolation space borders.
• Criteria: Here, Gaussian function is demonstrated. Given two molecules x
termined as below:
ϕ
parameter s. The cut off value associated with Gaussian potential functions, namely f
by methods based on sample percentile (Jouan-Rimbaud et al., 1999):
1
=
π
2
( )
where, Ф (x
s
.exp
( )
and xj) is the potential induced on xj by xi and width of the curve is defined by smoothing
i
−
1
2
s x x
2
( )
−
i j
(8)
2
and xj, it can be de-
i
can be calculated
p
196
EBSCOhost - printed on 2/14/2023 7:16 AM via . All use subject to https://www.ebsco.com/terms-of-use
( )
100
Importance of Applicability Domain of QSAR Models
f f q j f f
= + −
p i j j
with
( )
q pn= ×
−
+1
(9)
, where p is the percentile value of probability density, n is the number of compounds
in the training set and j is the nearest integer value of q. Test compounds with potential function values
lower than the set threshold are considered outside the AD.
5. RANGE OF THE RESPONSE VARIABLE
Based on the distribution plot or considering the range of the response variable, compounds can be identified as outside the AD. In a series of compounds, if the response value of a particular test compound
is largely different from the mean response value of the training set compounds, then the compound can
be considered as out of the AD.
6. MISCELLANEOUS APPROACHES
Along with the above-mentioned strategies to determine the AD of constructed QSAR models, there are several miscellaneous approaches introduced in the recent times. Here, we have tried to discuss these approaches
briefly. The use of these approaches has been very limited compared to the classical AD methods. The belowmentioned methods are especially used for interpolation space characterization in the model descriptor space.
6.1 Stepwise Based Approach
• Theory: Stepwise based approach for evaluating the AD of a QSAR model is proposed by Dimitrov
et al. (2005). This approach is useful for reflecting mechanistic rationality and transparency of the
QSAR model. This approach is processed in four stages in a chronological manner. The first stage
“General requirements domain” consists of checking of the variation of the physicochemical properties of the query chemicals in the training set compounds. The second stage “Structural domain”
defines the structural similarity which is found within the chemicals that are correctly predicted
by the model. The third stage “Mechanistic domain” takes into account of the mechanistic interpretation of the modeled phenomenon to define the domain of applicability. In this third step, the
model domain merges the reliability of specific functional groups assumed to cause the effect and
the domain of explanatory variables. Here, the latter is defined by the parameter interpolation
space. In the fourth and final stage “Domain of metabolic stimulator”, if stimulated metabolism of
chemicals is a part of the developed QSAR model then the dependability of simulated metabolism
is considered in assessing the reliability of the predictions.
• Criterion: An external compound is compulsory to satisfy all the conditions specified within the
mentioned four stages to be considered within the AD. The approach is considered as a rigorous
one as a chemical is assessed for similarity, metabolic and mechanistic check to address the reliability of predictions and permitting a better assessment of model’s AD (Dimitrov et al., 2005).
EBSCOhost - printed on 2/14/2023 7:16 AM via . All use subject to https://www.ebsco.com/terms-of-use
197
Importance of Applicability Domain of QSAR Models
• Drawback:
◦ In order to determine the AD of the QSAR model, one would need a lot of information re-
garding the structural attributes of compounds, mechanism of action for particular response
as well as their metabolic activation. In regular practice of developing QSAR models, this is
lacking in major cases.
◦ The method is time consuming as well as critical as a lot of variables are involved in this AD
assessment approach.
6.2 Decision Trees and Decision Forests
This approach identifies the AD in terms of prediction confidence and domain extrapolation based
on the consensus prediction of Decision Trees (DT) and Decision Forests (DF). The basic principle
is to minimize the overfitting which can be attained by merging the DTs and keeping the differences
within different DTs to maximum possible. Predictions from all the combined DTs are averaged in
order to find the prediction confidence for a particular molecule, while domain extrapolation provides
the prediction precision for that compound outside the training space (Netzeva et al., 2005; Tong et
al., 2003, 2004).
6.3 Kernel-Based Applicability Domain
Machine learning tools are becoming an important method to evaluate the molecular activities/
properties without the help of in vitro experiments. The major drawback of these methods is that
most of them do not provide any information using the knowledge contained in the model though
they are predicted adequately by the model. As a result, in machine learning based QSAR modeling, the estimation of the reliability of a model-based prediction is an important question (Fechner
et al., 2009).
Most machine learning approaches for QSAR heavily depend on a vectorial representation of
the molecules. Thus, the applicability domain is expressed as a subspace of the vector space with
one dimension for individual descriptor used. However, this vectorial concept cannot be directly
applied to kernel-based techniques like support vector machines. So, these methods have to rely on
an implicit feature space which is only defined by the applied kernel similarity and with unknown
dimensions. Therefore the domain of applicability of a kernel-based model has to be defined by
means of the kernel. This also permits us to employ the structured similarity measures like the
Optimal Assignment Kernel and its extension, instead of a numerical encoding. This method makes
it possible to depict the complex chemical structure of many drugs better than the methods which
are processed by using descriptors.
The concept of kernel density estimation has incorporated additional information in a trained
model. The added information can be achieved by using a weighted average kernel similarity of
a predicted molecule to the training data set. The weights can be attained either by utilizing the
knowledge contained in the learned model or by methods that describe the feature space structure
using the kernel (Fechner et al., 2009).
198
EBSCOhost - printed on 2/14/2023 7:16 AM via . All use subject to https://www.ebsco.com/terms-of-use
Importance of Applicability Domain of QSAR Models
6.4 Intelligent K-Means Clustering
An intelligent version of the k-means clustering algorithm-based measure of distance-to-domain for
QSAR models was designed by Stanforth et al., (2007) to obviate the difficulties of the existing methods (like convex hull, bounding box etc.) in terms of primitiveness and computational complexity. This
measure combines the modeling of the training set as a collection of the intelligent k-means clusters
algorithm in the descriptor space with a new interpretation of a conventional optimization criterion in
fuzzy clustering which leads to a modified harmonic mean measure. A test compound is assigned fuzzy
membership of each individual cluster, from which an overall distance may be calculated. Stanforth et
al., (2007) demonstrated that this approach is more stable than existing methods and more indicative of
the prediction error.
k-Means clustering is prepared based on the approximation of each point in a dataset by the centroid
of that point’s cluster. This estimation can be quantified by decomposing the data scatter into contributions that are explained and unexplained by the cluster model as presented in the following equation
(Mirkin, 2005):
k
x x N c c x c x c
⋅ = ⋅ + −
∑ ∑ ∑∑
i
∈ = ∈=
i T
i k
k
k k i k i k
1 1
In Equation (10), {x
k
( )
i Ck
k
: i ϵ T} is the training set and the k clusters Ck have sizes Nk and centroids ck,
i
⋅ −
( )
. (10)
respectively. This suggests an optimization of the so-called square-error k-means criterion
k
x c x c
−
( )
i k i k
∈=∑∑1
i Ck
k
⋅ −
( )
. (11)
Equation 11 holds for any inner product in the descriptor space.
k-Means clustering requires an initialization step to indicate the number of clusters k and the preliminary positions of their centroids. The initialization of the intelligent k-means algorithm has been
achieved by the principle of Anomalous Pattern Clustering (APC) (Mirkin, 2005; Smellie, 2004). The
principle can be demonstrated in the following way.
1. Two initial centroids are specified as follows: the first one is considered as the data gravity centre
where components are grand means of the corresponding components in the entire dataset, and
the second is an entity which is the furthermost away from the gravity centre.
2. It is interesting to point out that the first point (grand mean) centroid never changes, so that the
only changeable centroid is the second one.
The computational complexity proved to be quite acceptable for this method. Assessment of the
domain of applicability using clusters affords a much faster and easier calculation than k-NN. Not only
that, this one is more prudent in terms of additional stored model parameters, without negotiating the
acuity of the measure.
EBSCOhost - printed on 2/14/2023 7:16 AM via . All use subject to https://www.ebsco.com/terms-of-use
199
Importance of Applicability Domain of QSAR Models
PRACTICAL EXAMPLES OF DETERMINATION OF AD
In order to show an example of determination of the AD for a regression based QSAR model, we have
shown in Table 2 a set of hypothetical X-Y data and a correspoding QSAR model developed with multiple linear regression (MLR) tool. The model consists of three descriptors. Here, we have shown the
calculation and determination of parameters most commonly used in simple AD approaches like fixed or
probabilistic boundaries (bounding box), range of the response variable, leverage method and Euclidean
distance based method. All these methods can be used by any QSAR researchers without help of any
professional software.
Table 2. A simple example demonstrating calculation of applicability domain for a regression-based
QSAR model using different methods such as fixed or probabilistic boundaries (bounding box), leverage
method, and Euclidean distance based approaches
Compound
ID
1 3.45 1.65 1 1.65 3.46 0.41 -0.01 0.01 9.13 0.83 0.68
3 3.14 1.63 1.5 1.09 2.73 0.17 0.41 2.01 5.54 0.50 0.02
5 2.82 2.02 1.5 1.35 2.67 0.26 0.15 0.76 6.37 0.58 0.17
7 2.64 2.05 1.5 1.37 2.67 0.29 -0.03 -0.08 6.59 0.60 0.21
9 2.29 1.83 1.5 1.22 2.70 0.17 -0.41 -1.94 5.56 0.51 0.02
10 2.2 1.9 2 0.95 1.97 0.38 0.23 1.15 7.70 0.70 0.41
11 2.15 1.33 2 0.67 2.06 0.50 0.09 0.46 9.12 0.83 0.68
13 1.74 1.54 2 0.77 2.03 0.34 -0.29 -1.36 7.99 0.73 0.47
15 3.45 1.91 1 1.91 3.42 0.66 0.03 0.19 10.87 0.99 1.00
16 2.51 1.66 1.5 1.11 2.73 0.16 -0.22 -1.01 5.47 0.50 0.00
17 2.87 1.1 1.5 0.73 2.82 0.66 0.05 0.31 9.34 0.85 0.72
Compound
ID
2 3.2 1.9 1 1.9 3.42 0.39 -0.22 -1.02 10.78 0.98 0.99
4 2.87 1.22 1.5 0.81 2.80 0.33 0.07 0.40 8.32 0.76 0.53
6 2.81 1.78 1.5 1.18 2.71 0.14 0.10 0.53 5.51 0.50 0.01
8 2.49 1.61 1.5 1.07 2.74 0.16 -0.25 -1.14 5.67 0.52 0.04
12 2.01 1.96 2 0.98 1.96 0.30 0.05 0.28 7.92 0.72 0.45
14 3.51 1.88 1 1.88 3.42 0.38 0.09 0.46 10.62 0.97 0.96
*Y is the response variable; X
from equation 1, bcalculated from equation 2 and ccalculated from equation 3.
Y*
(Observed)
Y*
(Observed)
X1* X2* X3* Y
* X2* X3* Y
X
1
, X2 and X3 are the descriptors involved in the QSAR model, h is defined as the leverage value, acalculated
1
(Calculated)
(Predicted)
haResidual Standardized
Residual
Test Set
h Residual Standardized
Residual
Distance
b
Score
Distance
Score
Mean
Distance
Mean
Distance
c
Normalized
Mean
Distance
Normalized
Mean
Distance
200
EBSCOhost - printed on 2/14/2023 7:16 AM via . All use subject to https://www.ebsco.com/terms-of-use
sidual value Average sidual value
Importance of Applicability Domain of QSAR Models
1. Leverage Approach: Based on the calculated leverage (h) and standardized residual values, we
have constructed Williams plot (Figure 10). Here, the standardized residual (SR) value is calculated
based on the following equation:
Re Re
SR
=
S dard dev
tan
−
iiation of sidual value
Re
Training
( )
Training
( )
. (12)
Then Williams plot is constructed taking standardized values in Y-axis and leverage values in X-axis.
As the number of descriptors is three and number of compounds in the training set is eleven, the critical
leverage (h*) value is = [3*(3+1)]/11=1.09. Taking into consideration the criteria described in section
3.1, one can firmly conclude that based on the leverage approach for the studied QSAR model, all the
query compounds are within the AD.
2. Euclidean Distance-Based Approach: First, Euclidean distance scores are calculated (based on
Equation 2) for the total set of compounds and then the mean distance scores (based on Equation
3) are calculated. The mean distances are then normalized within the interval of zero to one. From
the normalized mean distances, it is clear that not a single test compound resides outside of the AD.
The Euclidean plot (Figure 11) is then developed taking the mean normalized distance in Y-axis
and compound numbers in X-axis.
3. Fixed or Probabilistic Boundaries (Bounding Box) and Range of the Response Variable: Both
approaches are strictly dependent on the boundary created by independent and response variables
of the training set compounds. Scrutinizing the descriptor values for the training and test sets, one
can easily identify that not a single test set compound falls outside the AD boundary created by
the independent variables. On the contrary, only one compound (14) can be considered outside of
the AD boundary when someone is taking in consideration the response variable range. Therefore,
one can conclude that considering both methods, only one compound (14) resides outside the AD
though it is lying very close to the boundary created by response variable.
While taking into consideration the above mentioned methods for the identification AD for a single QSAR
model, one may remember that no single method of AD determination can be considered as the universally
the best one. It may also be noted that the results of one specific method may not match with those from another approach as each method is based on different statistical algorithm for the determination of AD. Thus,
multiple methods should be applied before coming to a final conclusion regarding AD of a QSAR model.
FUTURE RESEARCH DIRECTION
It is generally believed that no single AD approach can be relied as universally ideal one to identify the
interpolation region for any QSAR model. Extensive and fruitful research is being carried out on the development of some new approaches or modification of already existing approaches to make them more acceptable and reliable ones. The following points should be considered in future research regarding AD studies.
EBSCOhost - printed on 2/14/2023 7:16 AM via . All use subject to https://www.ebsco.com/terms-of-use
201
Importance of Applicability Domain of QSAR Models
Figure 10. Williams plot for the arbitrarily developed QSAR model
Figure 11. Euclidean distance plot for the arbitrarily developed QSAR model
202
EBSCOhost - printed on 2/14/2023 7:16 AM via . All use subject to https://www.ebsco.com/terms-of-use
Importance of Applicability Domain of QSAR Models
1. A global similarity test should be developed to identify whether the structural features of a new
external test compound are enclosed in the training set of compounds from which the QSAR model
is developed.
2. There should be a conceptual framework for the assessment of AD for any QSAR model by the
regulatory agencies and QSAR experts as no further modification has been done in the OECD
guidelines after its establishment.
3. Some effort is needed to investigate the acceptable confidence limits for various AD approaches.
Confidence limits could be a useful addition to the AD to predict the new set of compounds more
confidently.
4. In order to make the AD estimation more user friendly, QSAR developers have to build up auto-
mated software tools where one can easily develop interpolation space as well as scatter plots for
respective AD study.
5. There is a need of proper awareness of the AD concept and its implementation in the development
of QSAR models. Without a proper AD study, no QSAR model should be accepted by the journals
as well as regulatory agencies, as acceptability of any QSAR model is decidedly related with AD.
6. The QSAR model development and the assessment of AD (regarding algorithm and interpretation)
should be transparent and it is the major responsibility of the model developer and the regulatory
agency to make it clear to the readers and QSAR learners.
CONCLUSION
Applicability domain of a QSAR model is estimated by determining interpolation regions as defined
by the training data set in model descriptor space but the region varies depending on the implemented
approach. It is interesting to point out that all the implemented AD approaches have their own potential
and weakness. Few methods have complex algorithms behind the identification of interpolation regions,
on the contrary some have strong and easy statistical background behind the AD estimation. Therefore,
it is totally dependent on the model developer how he/she chooses the specific approach according to
his/her need to define the applicability domain for the model more accurately and in robust way.
The basic principle of any AD approach is to find out the number of test set compounds which are
falling outside of the defined domain of applicability. It is important to note that if someone uses different AD approaches for developed QSAR model from a single dataset, the result may vary in a big
way and none of these established approaches can be considered universal one. Therefore, it is always
recommended to assess the results from different available possible strategies before assessing a new
compound set to get the result with full confidence.
Distance-based approaches are largely used over the other approaches by QSAR developers in recent
times. In case of other approaches, the number of test compounds outside the AD is much higher as they
are concentrated towards the training set extremities. The distance from training space is significant in
defining the model’s AD for test set compounds. Again, bad predictions for test compounds are also
considered as outside the AD with most of the approaches. The thresholds values are also not fixed for
many approaches which may lead to vary the number of compounds outside the AD. So, these issues
should be considered more carefully by the QSAR fraternity and regulatory bodies to make the AD
studies more transparent and robust ones.
EBSCOhost - printed on 2/14/2023 7:16 AM via . All use subject to https://www.ebsco.com/terms-of-use
203
Importance of Applicability Domain of QSAR Models
The following points should be considered by any QSAR model developer while considering AD
studies to make his/her QSAR model more robust, predictive and reproducible:
1. The utmost important aspect of any QSAR model is its prediction capability for new set of compounds. The AD study has emerged as one of the key and compulsory validation criteria considering
the reliability of predictive capability of QSAR model for a new set of compounds.
2. Considering the basic principle of AD approaches, one has to rely on the AD of the QSAR before
estimating activity/property/toxicity of any chemical or pharmaceuticals by the developed QSAR
model.
3. The core of AD for any specific QSAR model is the training space which is constructed by the
training set of compounds. Therefore, the selection of the training set is very crucial aspect as it
should consist of the characteristics of the total dataset to reflect the effect of each compound in
the developed model.
4. The response value or endpoint of the QSAR model is determined experimentally. So, the reliability
of these values (experiments should be performed in same condition for same endpoints) should
be thoroughly checked before constructing the QSAR models. As AD is also dependent on the
developed QSAR model, error in response value will mislead the QSAR model which will lead to
erroneous interpolation space for AD.
5. There is always a chance of uncertainty related with the assessment of AD for any QSAR model. If
the constructed QSAR model is not reliable, one cannot get confidence in AD assessment. Again,
mechanistic aspects are rarely considered in the model development by the modeler to make the
fact more uncertain one.
6. Estimation of AD by various approaches is highly dependent on a number of issues like the model
dimensionality, used descriptors, response value as endpoints, underlying data distribution and last
but not the least algorithm of AD determination. So, there is always a chance of large number of
assumptions in the AD study.
REFERENCES
Bender, A., & Glen, R. C. (2004). Molecular similarity: A key technique in molecular informatics. Journal
of Organic and Biomolecular Chemistry, 2(22), 3204–3218. doi:10.1039/b409813g PMID:15534697
Benigni, R., & Giuliani, A. (2003). Putting the predictive toxicology challenge into perspective: Reflections on the results. Bioinformatics (Oxford, England), 19(10), 1194–1200. doi:10.1093/bioinformatics/
btg099 PMID:12835261
Carlsen, L., Kenessov, B. N., & Batyrbekova, S. Y. (2009). A QSAR/QSTR study on the human health
impact of the rocket fuel 1, 1-dimethyl hydrazine and its transformation products: Multicriteria hazard
ranking based on partial order methodologies. Environmental Toxicology and Pharmacology, 27(3),
415–423. doi:10.1016/j.etap.2009.01.005 PMID:21783973
204
EBSCOhost - printed on 2/14/2023 7:16 AM via . All use subject to https://www.ebsco.com/terms-of-use
Importance of Applicability Domain of QSAR Models
Chen, J. J., Tsai, C. A., Young, J. F., & Kodell, R. L. (2005). Classification ensembles for unbalanced
class sizes in predictive toxicology. SAR and QSAR in Environmental Research, 16(6), 517–529.
doi:10.1080/10659360500468468 PMID:16428129
Dimitrov, S., Dimitrova, G., Pavlov, T., Dimitrova, N., Patlewicz, G., Niemela, J., & Mekenyan, O. A.
(2005). Stepwise approach for defining the applicability domain of SAR and QSAR models. Journal of
Chemical Information and Modeling, 45(4), 839–849. doi:10.1021/ci0500381 PMID:16045276
Eriksson, L., Jaworska, J., Worth, A. P., Cronin, M. T., McDowell, R. M., & Gramatica, P. (2003).
Methods for reliability and uncertainty assessment and for applicability evaluations of classificationand regression-based QSARs. Environmental Health Perspectives, 111(10), 1361–1375. doi:10.1289/
ehp.5758 PMID:12896860
Fechner, N., Jahn, A., Hinselmann, G., & Zell, A. (2009). Atomic local neighborhood flexibility incorporation into a structured similarity measure for QSAR. Journal of Chemical Information and Modeling,
49(3), 549–560. doi:10.1021/ci800329r PMID:19434895
Ferreira, M. M. C. (2001). Polycyclic aromatic hydrocarbons: A QSPR study. Chemosphere, 44(2),
125–146. doi:10.1016/S0045-6535(00)00275-7 PMID:11444294
Forina, M., Armanino, C., Leardi, R., & Drava, G. (1991). A class-modelling technique based on potential
functions. Journal of Chemometrics, 5(5), 435–453. doi:10.1002/cem.1180050504
2
Golbraikh, A., & Tropsha, A. (2002). Beware of q
! Journal of Molecular Graphics & Modelling, 20(4),
269–276. doi:10.1016/S1093-3263(01)00123-1 PMID:11858635
Gramatica, P. (2007). Principles of QSAR models validation: Internal and external. QSAR & Combinato-
rial Science, 26(6), 694–701. doi:10.1002/qsar.200610151
Guha, R., & Jurs, P. C. (2005). Determining the validity of a QSAR model-a classification approach.
Journal of Chemical Information and Modeling, 45(1), 65–73. doi:10.1021/ci0497511 PMID:15667130
Hair, J. F. Jr, Anderson, R. E., Tatham, R. L., & Black, W. C. (2005). Multivariate data analysis. Singapore: Pearson Education.
Jaworska, J., Nikolova-Jeliazkova, N., & Aldenberg, T. (2005). QSAR applicability domain estimation
by projection of the training set descriptor space: A review. Alternatives to Laboratory Animals, 33(5),
445–459. PMID:16268757
Jaworska, J. S., Comber, M., Auer, C., & Van Leeuwen, C. J. (2003). Summary of a workshop on regulatory acceptance of (Q)SARs for human health and environmental endpoints. Environmental Health
Perspectives, 111(10), 1358–1360. doi:10.1289/ehp.5757 PMID:12896859
Jaworska, J. S., Nikolova-Jeliazkova, N., & Aldenberg, T. (2004). Review of methods for applicability
domain estimation. Report. Ispra, Italy: The European Commission-Joint Research Centre.
Jouan-Rimbaud, D., Bouveresse, E., Massart, D. L., & de Noord, O. E. (1999). Detection of prediction
outliers and inliers in multivariate calibration. Analytica Chimica Acta, 388(3), 283–301. doi:10.1016/
S0003-2670(98)00626-6
EBSCOhost - printed on 2/14/2023 7:16 AM via . All use subject to https://www.ebsco.com/terms-of-use
205