Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_3091_Библиотеки_им_академика_М_И_Перельмана
.pdf
96
22. Penn L, White M, Lindstrom J, den Boer AT, Blaak E, Eriksson JG, Feskens E, Ilanne-Parikka
P, Keinanen-Kiukaanniemi SM, Walker M, Mathers JC, Uusitupa M, Tuomilehto J (2013)
Importance of weight loss maintenance and risk prediction in the prevention of type 2 diabe-
tes: analysis of European Diabetes Prevention Study RCT. PLoS ONE 8:e57143.
https://doi.
org/10.1371/journal.pone.0057143
23. Ramachandran A, Snehalatha C, Ram J, Selvam S, Simon M, Nanditha A, Shetty AS,
Godsland IF, Chaturvedi N, Majeed A, Oliver N, Toumazou C, Alberti KG, Johnston DG
(2013) Effectiveness of mobile phone messaging in prevention of type 2 diabetes by lifestyle
modication in men in India: a prospective, parallel-group, randomized controlled trial. Lancet
1:191–198
24. Tuomilehto J, Lindström J, Eriksson JG, Valle TT, Hämäläinen H, Ilanne-Parikka P, Keinänen-
Kiukaanniemi S, Laakso M, Louheranta A, Rastas M, Salminen V, Uusitupa M, Finnish
Diabetes Prevention Study Group (2001) Prevention of type 2 diabetes mellitus by changes in
lifestyle among subjects with impaired glucose tolerance. New Engl J Med 344:1343–1350
25. Lytle LA (2009) Examining the etiology of childhood obesity: the IDEA study. Am J
Community Psychol 44:338–349
26. Mendes de Leon C, Powell LH, Kaplan BH (1991) Change in coronary-prone behaviors in the
Recurrent Coronary Prevention Project. Psychosom Med 53:407–419
27. Jones DA, West RR (1996) Psychological rehabilitation after myocardial infarction: multicen-
tre randomised controlled trial. BMJ 313:1517–1521
28. Robinson TN, Matheson DM, Kraemer HC, Wilson DM, Obarzanek E, Thompson NS,
Alhassan S, Spencer TR, Haydel KF, Fujimoto M, Varady A, Killen JD (2010) A random-
ized controlled trial of culturally tailored dance and reducing screen time to prevent weight
gain in low-income African American girls: Stanford GEMS. Arch Pediatr Adolesc Med
164:995–1004
29. The Trials of Hypertension Prevention Collaborative Research Group (1992) The effects of
nonpharmacologic interventions on blood pressure of persons with high normal levels. Results
of the Trials of Hypertension Prevention, Phase I.JAMA 267:1213–1220
30. Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB (2013) Designing clinical
research, 4th edn. Lippincott Williams & Wilkins, Philadelphia
31. The Diabetes Prevention Program (DPP) Research Group (1999) The Diabetes Prevention
Program. Design and methods for a clinical trial in the prevention of type 2 diabetes. Diabetes
Care 22:623–634
32. The, Diabetes Prevention Program (DPP) Research Group (2002) The Diabetes Prevention
Program (DPP): description of lifestyle intervention. Diabetes Care 25:2165–2171
33. Diabetes Prevention Program Research Group (2009) 10-year follow-up of diabetes incidence
and weight loss in the Diabetes Prevention Program Outcomes Study. Lancet 374:1677–1686
34. Howard BV, Van Horn L, Hsia J, Manson JE, Stefanick ML, Wassertheil-Smoller S, Kuller
LH, LaCroix AZ, Langer RD, Lasser NL, Lewis CE, Limacher MC, Margolis KL, Mysiw
WJ, Ockene JK, Parker LM, Perri MG, Phillips L, Prentice RL, Robbins J, Rossouw JE, Sarto
GE, Schatz IJ, Snetselaar LG, Stevens VJ, Tinker LF, Trevisan M, Vitolins MZ, Anderson
GL, Assaf AR, Bassford T, Beresford SA, Black HR, Brunner RL, Brzyski RG, Caan B,
Chlebowski RT, Gass M, Granek I, Greenland P, Hays J, Heber D, Heiss G, Hendrix SL,
Hubbell FA, Johnson KC, Kotchen JM (2006) Low-fat dietary pattern and risk of cardiovas-
cular disease: the Women's Health Initiative randomized controlled dietary modication trial.
JAMA 295:655–666
35. The Trials of Hypertension Prevention Collaborative Research Group (1997) Effects of weight
loss and sodium reduction intervention on blood pressure and hypertension incidence in over-
weight people with high-normal blood pressure. The Trials of Hypertension Prevention, Phase
II. Arch Intern Med 157:657–667
36. Duncan E, O’Cathain AO, Rousseau N, Croot L, Sworn K, Turner K, Yardley L, Hoddinott
P (2020) Guidance for reporting intervention development studies in health research
(GUIDED): an evidence-based consensus study. BMJ Open 10:e033516. https://doi: 10.1136/
bmjopen-2019-033516
4 Hypothesized Pathway andBias
https://t.me/medicina_free
97© Springer Nature Switzerland AG 2021
L. H. Powell et al., Behavioral Clinical Trials for Chronic Diseases,
https://doi.org/10.1007/978-3-030-39330-4_5
Chapter 5
Clinical Signicance
“If this comment was hardly original in 1966, it can hardly be original now. Yet
this naked emperor has been shamelessly running around for a long time.”
Cohen, 1994 [2]
This chapter rst considers the strengths and limitations of statistical inference.
Its limitations are often overlooked relative to its convenience and immense pop-
ularity as a decision-making metric. This background provides the basis for the
argument that statistical signicance is not the only, or the best, judge for mak-
ing the case that a behavioral treatment can move a chronic disease endpoint.
More persuasive is a demonstration of clinical signicance where a behavioral
treatment is strong enough to neutralize the risk incurred by a behavioral risk
factor. The second part of the chapter is directed toward how to identify clini-
cally signicant treatment targets, incorporate clinical signicance into the anal-
ysis of trial results, and integrate clinical signicance into the calculation of
sample size. The contribution of any single behavioral trial is assessed by the
“To say that a great deal of mischief has been associated with the test of
signicance is hardly original. It is what ‘everybody knows.’. . . . To say it ‘out
loud’ is to assume the role of the child who pointed out that the emperor was
really outtted in his underwear.”
Bakan 1966 [1]
Fundamental Point
In a behavioral clinical trial, change in the behavioral target of treatment
must be strong enough to test the hypothesis that improving it produces a
benecial effect on a clinical outcome. Judgment of the strength of a behav-
ioral treatment should be based more heavily on clinical, than statistical,
signicance. Clinical signicance can be used to identify treatment targets,
analyze trial results, and calculate sample size. By so doing, the strength of
scientic conclusions that are possible from any single behavioral clinical
trial can be enhanced.
https://t.me/medicina_free
98
strength of the scientic conclusions drawn from it. This includes a clinically
signicant benet of treatment, quantied certainty that this benet is greater
than a relevant comparator, biologic plausibility, and consistency with results
from past studies. The conclusion that a behavioral treatment improves health is
best answered progressively, through replication.
SCIENTIFIC PRINCIPLES
A “signicant” result from a trial refers to an “important” result. A “statistically
signicant” result from a trial evaluates “importance” by determining the likelihood
that an observed result would occur if the reality was that the treatment had no ben-
et. A “clinically signicant” result from a trial evaluates “importance” by whether
or not a result is large enough to be meaningful and useful to patients, providers, or
other stakeholders.
Statistical signicance is appealing because it is quantied, evaluated against a con-
ventional standard in the form of the null hypothesis, and provides a simple, cate-
gorical answer. Clinical signicance has less practical appeal because it is more
subjective, context-specic, and multiply-determined. In a behavioral clinical trial
aimed at improving a chronic disease endpoint, the value of the behavioral treat-
ment is whether or not it is strong enough to reduce behavioral risk to a level suf-
cient to move this endpoint. Since this criterion is one of clinical signicance, a
“signicant” result in a behavioral trial is a result that is clinically signicant.
Inferential Statistics
The logic behind behavioral trials, drug trials, or any experiment is based upon
inductive reasoning: an inference from an observation in a specic experiment to a
general state of affairs. Although this logic is part of the fabric of science, it is fal-
lible. Hume was the rst to challenge inductive logic by questioning what makes it
possible to extrapolate from a limited sample to a broader generalization [3]. Karl
Popper, the great philosopher of science, devoted the early phase of his career to this
challenge and concluded that it was logically impossible to generalize with cer-
tainty from an observation in a sample to a population [4].
This logical problem puts the foundation of science on shaky ground with no simple
solution. It did, however, serve as a catalyst for innovation among statisticians of the
1920s and 1930s. These innovations provide the foundation of classical inferential
statistics.
5 Clinical Signicance
https://t.me/medicina_free
99
Statisticians, most notably Fisher, Neyman, and Pearson, accepted the philosophical
argument that inductive inferences about a scientic hypothesis could never be
made with certainty. So they proposed instead to use a statistical hypothesis that
featured deductive inferences made from the general (i.e., the null hypothesis) to the
specic (i.e., the experiment). Using this framework, they developed methods by
which to quantify the risk of error in making such deductive inferences. Fisher
articulated the approach in this way:
“Any inference must be attended with some degree of uncertainty. But this is not the
same as to admit that such inference cannot be absolutely rigorous. For the nature and
degree of uncertainty may itself be capable of rigorous expression.” [5]
To quantify the risk of being wrong, the statisticians drew on two basic principles.
The rst is the frequentist principle. This assumes that characteristics of a popula-
tion have an inuence on a sample drawn from it. Therefore, dramatic departure
from population characteristics makes it likely that a sample is from a different
population [1]. The second principle is once-ness. This assumes that experiments
are carried out with limited resources so methods need to offer a shortcut around the
more time-intensive demands of science for repeated replication [1, 5].
The P-Value
Fisher’s key contribution to modern statistical approaches was the “probability
value” (p-value). Based upon the frequentist principle, the p-value, calculated after
the experiment has been conducted, is the probability that the obtained result would
occur if the null hypothesis of no-treatment effect was true. Convention has con-
verged upon a p-value of ≤0.05, based upon the rationale that an event occurring
only 5 times out of 100 is unlikely enough to be inconsistent with the null hypoth-
esis of no treatment benet. Since the p-value is calculated on the assumption that
the null hypothesis is true, it does not provide a direct measure of the null hypoth-
esis being false or that any alternative hypothesis is true [6]. Fisher argued that the
p-value is just a simple tool, to be used only as one part of a uid qualitative process
of drawing a conclusion from observed data [7], consistent with the approach origi-
nally proposed by Bayes [8]. In Fisher’s words:
“. . . no scientic worker has a xed level of signicance at which from year to year, and
in all circumstances, he rejects hypotheses; he rather gives his mind to each particular
case in the light of his evidence and his ideas.” [9]
The Hypothesis Test
The p-value provides the basis for an inference, not an action. If the null hypothesis
is rejected, what is the correct course of action? This was the problem for which
Neyman and Pearson developed a solution in the form of statistical hypothesis testing
[10]. They applied the once-ness principle and found a shortcut around dependence
on replication by developing a decision-making approach which produced a long-
The Scientic Principles
https://t.me/medicina_free
100
term error rate that could be applied to statistical conclusions from any one experi-
ment. Two hypotheses were posed: a null hypothesis of no effect and an alternative
hypothesis of an effect. Accepting one required the rejection of the other. Two types
of errors result: falsely concluding that the alternative hypothesis is true (Type I error)
and falsely concluding that the null hypothesis is true (Type II error). It is critical to
keep in mind that this error rate does not apply to a single experiment but to the error
that would occur over many replications of the experiment. The acceptable error rate
is built into the design of the trial before any result is obtained. In their own words:
“Without hoping to know whether each separate hypothesis is true or false, we may
search for rules to govern our behavior with regard to them, in following which we insure
that, in the long run, we shall not often be wrong.” [10]
These two approaches to statistical inference can be understood by borrowing an
analogy from the legal system. P-values quantify the likelihood of making an error
in convicting an innocent defendant. Hypothesis testing quanties the overall rate of
errors in convictions made by the court.
The Condence Interval
Derived from the same frequentist mathematics as the p-value, the condence inter-
val represents the range of values that 95 times out of 100 would contain the true
population parameter if an experiment were repeated many times. Because the
emphasis is on the range of values possible for an observed effect, it provides more
precise information about the degree of uncertainty and, as such, is gaining increas-
ing acceptance in high-visibility journals. Although a step in the right direction, they
are still used as surrogates for categorical p-values by looking to see if a null effect
is contained within the interval. They do, however, provide an opportunity to deter-
mine a priori what a clinically signicant effect would be and to reference the inter-
val to see if it is contained within it. If it is, then a clinically signicant benet is a
plausible result of the treatment [6].
Limitations ofInferential Statistics
Inferential statistics have become the preeminent criteria for decision-making in the
scientic literature. But in such diverse disciplines as statistics, medicine, psychol-
ogy, and business, and dating as far back as the time when these procedures were
rst introduced [11], criticisms abound. Most recently, the American Statistical
Association (ASA) issued a statement on p-values and statistical signicance. The
rationale for this statement was that decades of saying what not do have gone mostly
unheeded. In their words:
“It is as though statisticians were asking users of statistics to tear out the beams holding
up the edice of modern scientic research without offering solid materials to replace
them. Pointing out old rotting timbers was a good start, but we now need more.” [12]
5 Clinical Signicance
https://t.me/medicina_free
101
In their most recent statement,
they have reiterated the “Don’t’s”
articulated over the past decades.
But now, in hopes of action, they
have accompanied them with a list
of “Do’s” (see the box). The
“Don’t’s” and “Do’s” that are par-
ticularly pertinent for behavioral
clinical trials can be summarized
into the four key points presented
in the box.
Misinterpretation
What we want to know is the prob-
ability that a scientic hypothesis
is true. What we get from a p-value
is the probability that the null
hypothesis, created with some
assumptions, is true. It is a logical
error, referred to as proof by con-
tradiction [2], to infer that a low
probability event under the null
hypothesis is evidence that the sci-
entic hypothesis is true. To get
the needed information, it is
tempting to turn to hypothesis test-
ing. But what we get is an alpha
level, predetermined before any experiment is undertaken, that represents the level
of error in accepting the scientic hypothesis that could be expected over many tests
conducted over many repetitions. It says very little about error in any specic test.
It would be extremely convenient to have a single statistic that provides conclusive
support for a scientic hypothesis. But since no single statistic can accomplish this,
the interpretive error referred to as the hybrid p-value has arisen [2, 6]. Because the
p-value and the alpha level look alike, both often being 0.05, it is easy to combine
them into one hybrid statistic that quanties error made in a single test when infer-
ring that a research hypothesis is true. Making this error gives us what we want: the
probability that a scientic hypothesis is true. Fisher, Neyman, and Pearson argued
against this misinterpretation by saying that error cannot be quantied simultane-
ously as an event in the short run and a rate in the long run [6]. Beyond this interpre-
tative error, categorization of a p-value into an acceptable error of ≤0.05 provides a
convenient dichotomous decision rule: the treatment worked or it did not. But this
rule is based upon imsy and misleading evidence [13]. This misinterpretation has
had a profound inuence on the process of science by manufacturing convenient
scientic conclusions that are not restricted by the limits imposed upon either
p-values or hypothesis testing.
American Statistical Association
Statement On P-Values
and Statistical Signicance [12]
DON’T DO
Say “statistically
signicant”
Accept uncertainty:
seek replication
Base conclusions on
statistical
signicance
Be thoughtful:
recognize difference
between exploration
and conrmation
Believe the p-value
is the probability
that a scientic
hypothesis is true
Be open:
judge value by a
variety of criteria;
one trial is rarely
enough
Judge scientic
value by statistical
signicance
Be modest:
understand and
express clearly the
limitations of
science and of one’s
own work
The Scientic Principles
https://t.me/medicina_free
102
The ASA statisticians encouraged modesty in interpretations of results. This mod-
esty is expressed by understanding the limitations of inferential statistics, retiring
the term “statistical signicance,” reporting exact p-values, and replacing the term
“condence interval” with “compatibility interval” to show all the values that are
reasonably compatible with the data [12, 13].
False Security
Basing scientic conclusions on statistical signicance leads to a false sense of
security about results from a single trial. Although a statistical verdict has legiti-
macy because it is quantitative and requires minimal interpretation, it promotes a
tyranny of statistical evidence over such types of evidence as biologic plausibility
and consistency with past research [6, 14]. Moreover, acceptance of the statistical
verdict as denitive undercuts scientic progression, accumulation of knowledge,
and replications that are central to the condence that a treatment works [15]. The
falseness of this security has been revealed by a high rate of non-replication of nd-
ings in clinical trials. This is likely due, in part, to claims of conclusive ndings
from a single trial evaluated using statistical signicance criteria [16].
The ASA statisticians encouraged acceptance of uncertainty in making scientic
conclusions. Uncertainty is reected, in large part, in the drive to embrace variation
in the effects of a treatment and to view statistical results as being much more
incomplete and uncertain than is currently the norm [12].
Ease ofManipulation
One needs only to do the “sample size samba” several times to understand how easy
it is to manipulate statistical signicance by manipulating assumptions in the design
of a trial [17]. The dance is set in motion as investigators and biostatisticians go
back and forth varying design assumptions to get to a sample size that is within
resources to implement. When the sample size comes out to be too high, assump-
tions about the treatment effect, the choice of a primary outcome, the dropout rate,
and/or the stringency of the comparator are relaxed. This ease of manipulation
opens the door to a potentially dangerous state of affairs where the design of the trial
is informed by maximizing statistical signicance rather than answering a key
scientic question. This practice is reinforced by academic pressures for publica-
tions, combined with publication practices that seek papers with positive results.
The ASA statisticians encouraged thoughtfulness in the use of statistical proce-
dures. “Statistical thoughtfulness” is reected in sensitivity to the different objec-
tives of exploratory and conrmatory studies and nding the right method for a
question, rather than nding the right question for a method [12].
5 Clinical Signicance
https://t.me/medicina_free
103
Statistical Signicance Is Not Clinical Signicance
It has long been recognized that a result can be statistically, but not clinically,
signicant. That is, it can be statistically signicant but not very important to
patients, providers, or other stakeholders. Alternatively, a result can be clinically,
but not statistically, signicant because of such design manipulations as a small
sample size. (See Chapter 3: Behavioral Treatment Development, Figure 3.3, for
an example.) While statistical signicance shows the probability that an observed
result is from a null distribution, clinical signicance focuses on the potency of
treatment, whether it is large enough to move the chronic disease needle and
whether it will make a difference in patient’s lives [18]. The problem is that statis-
tical signicance has become a proxy for clinical signicance. In a systematic
sampling of 869 journals in general medicine, clinical specialties, epidemiology,
and public health, 81% equated statistical signicance with clinical signicance
[19]. This is likely due, in part, to the fact that it is generally easier to get a statisti-
cally signicant result than one that is clinically signicant [20].
Prioritizing statistical over clinical signicance in a Phase III behavioral trial is a
Pyrrhic victory. When nature is the nal arbiter, a short-term win from statistics sets
the stage from a longer-term defeat from nature. It would be a mistake to assume
that statistical signicance has no role in scientic investigation. Indeed, it serves as
a handrail that guides decisions and prevents researchers from getting carried away
by hasty, impressionistic generalizations [21]. But the balance between statistical
and clinical signicance needs to be shifted so that quantitative methods are better
integrated into the more general picture of clinical benet.
The AHA statisticians encouraged openness in making scientic conclusions. This
openness is expressed by judging conclusions in a particular trial not simply by a
p-value but by relevant prior evidence, plausibility of the mechanism, study design,
data quality, and the cost vs. benet ratio [12].
CHALLENGES FOR
BEHAVIORAL TRIALS
The ultimate rationale for conducting a Phase III behavioral efcacy trial is the
strength of the hypothesis that a behavioral treatment can produce a clinically sig-
nicant benet on an important chronic disease outcome. The rationale for conduct-
ing a Phase II behavioral efcacy trial is the strength of the hypothesis that a
behavioral treatment can produce a clinically signicant reduction in behavioral
risk and thereby neutralize the risk on the chronic disease endpoint which would be
otherwise incurred. This chain of reasoning is straightforward, but progress in
developing this rationale can be undercut in several ways.
Challenges forBehavioral Trials
https://t.me/medicina_free
104
“One-Size-Fits-All” Methods
Many applied behavioral scientists who develop interests in conducting behavioral
clinical trials were trained in the tradition of experimental psychology. This is a
tradition of discovery of mechanisms and how a treatment works. Experiments are
designed to test hypotheses about mechanisms, mediators, moderators, and compo-
nents. To make use of all of the data, predictors and outcomes tend to be continuous.
They are analyzed for statistical signicance using such procedures as correlations,
regressions, and path analyses [22].
But when the focus progresses to testing the efcacy of behavioral treatments on
chronic disease outcomes, the aim is to conrm and make decisions and the tradi-
tion shifts to clinical trial methodology. Hypotheses tend to compare a new treat-
ment to existing care. Predictor and outcome data tend to be categorical, featuring
clinically signicant benchmarks for judging both the success of a treatment and the
reduction in disease risk. Statistics of choice include rates, relative risks, and risk
differences. Analytic procedures are often time-to-event.
Clinical trial methodology, a synthesis of clinical research, biostatistics and epide-
miology, is part of the culture of medicine and the fabric of medical training, but it
is not a part of traditional training in the behavioral sciences. By the same token,
design of experiments, aimed at understanding the dynamics of behavioral treat-
ments, is a part of the culture of behavioral sciences but is not part of traditional
training in medicine or epidemiology. These training gaps come to a head when
conducting behavioral clinical trials. Behavioral treatments which have not been
optimized are tested by medical scientists with ripe and expensive clinical trial
methods. Optimized behavioral treatments are tested by behavioral scientists using
the experimental psychology methods of discovery and experimentation rather than
methods aimed at conrmation, decision-making, and efcacy. A translational sci-
ence model draws on methods from both of these traditions to link maturing ques-
tions about a behavioral treatment to an evolving, transdisciplinary set of methods.
Signicance Tests Are theStatus Quo
Signicance testing is woven into the fabric of most scientic investigation. But
psychology was one of the rst disciplines to embrace it with enthusiasm, shortly
after it was introduced by Fisher [15]. Use of signicance testing was reported in
10% of the empirical papers in psychology published in the early part of 1900 but
jumped to 90% of the papers published in the 1970s [23]. Signicance testing is far
more common in the social sciences than it is in such physical sciences as physics
and chemistry where many scientists regard it as unscientic [24]. Some have
referred to the extensive use of signicance testing as a ritual, similar to compulsive
hand washing [25]. In Bakan’s words:
5 Clinical Signicance
https://t.me/medicina_free
105
“Statistical hypothesis testing is profoundly woven into the psychological research enter-
prise, constituting a critical part of the total cultural-scientic tapestry. To pull out the
strand of the test of signicance would seem to make the whole tapestry fall apart.” [1]
Despite Fisher’s emphasis on the primitive nature of the p-value and the importance
of restricting its use to problems about which we know very little [26], signicance
testing is used to test all types of interventions, at all phases of development, ignoring
the mathematical and philosophical framework from which the test emerged.
While it is appealing to
have one standard answer
to all types of questions, it
runs the risk of providing a
conclusive answer to the
wrong question. If, for
example, interest is in
whether or not a behavioral
treatment is strong enough
to improve a clinical out-
come, statistical signi-
cance will provide a
conclusive answer but not
to that question. It is this
very problem that has
directed statisticians and clinical researchers toward Bayesian analyses which aim
to enhance the quality of scientic conclusions through the use of a methodology
that incorporates other ways of knowing beyond statistical signicance including
clinical signicance, biologic plausibility based upon potential biological mecha-
nisms, and past research [27, 28]. In the eyes of statisticians, signicance testing is
the bottom rung on a ladder of increasingly more informative hypothesis testing
methods progressing to empirical Bayes methodology [29].
OVERCOMING THESE
CHALLENGES INA
BEHAVIORAL TRIAL
A key question in developing and evaluating a behavioral treatment for a chronic
disease is whether the treatment is strong enough to reduce risk incurred by a behav-
ioral risk factor and, by so doing, improve a disease endpoint. This is a question of
clinical signicance. It is a categorical, yes/no decision. It requires a milestone for
making a decision about readiness to move to a Phase III efcacy trial [30].
Overcoming These Challenges ina Behavioral Trial
https://t.me/medicina_free
Соседние файлы в папке Библиотека им академика М.И. Перельмана