Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_3091_Библиотеки_им_академика_М_И_Перельмана
.pdf
76
Science is an imperfect method. Belief that the sun always rises stems from the fact
that it has done so every day in recorded history and the mechanism of the earth’s
rotation is understood. Belief that beta blockers are an effective treatment for high
blood pressure is strong because 147 clinical trials have shown benet and the
mechanisms are understood. Replication bolsters belief but does not provide a guar-
antee. If only one trial is available, belief should be, but often is not, consider-
ably weaker.
We must embrace uncertainty in science. In the face of an imperfect method, knowl-
edge is transitory, and humility is appropriate. In Duchamp’s words:
“Knowledge keeps like sh.”
Marcel Duchamp, 1917
CHALLENGES FOR
BEHAVIORAL TRIALS
A rigorous behavioral clinical trial is guided by a quantied hypothesis that is vul-
nerable to falsication and free of biases that would render results inconclusive.
There are two practices common in the current behavioral trials culture that pose a
challenge to this rigor.
Exploratory Aims inConrmatory Trials
A translational science model for the progression of research on behavioral treat-
ments for chronic diseases evolves from exploration and discovery in the early
phases to conrmation in the later phases. As relevant questions progress, so too do
the methods used to answer them. A challenge is posed for a behavioral trial when
there is an attempt to mix these separate questions together into one single trial.
It is a common practice to guide a behavioral trial by an inclusive and explanatory
conceptual model. These models represent a disease process by portraying a com-
plex web of causal factors, including bidirectional and reciprocal causal associa-
tions and mediators and moderators of these associations. These models can be
tested in experiments as well as observational studies using such methods as path
analysis to determine strength, direction, and independence of various paths. An
example of an explanatory conceptual model is portrayed in Fig.4.2 at the top.
Itportrays the complex web of interacting factors that explain childhood obesity [25].
The conceptual model is inclusive, portraying the array of immutable, contextual,
behavioral, and biological factors that together explain the problem of childhood
obesity. Paths to obesity are bidirectional and reciprocal. Assessments of the strength
4 Hypothesized Pathway andBias
https://t.me/medicina_free
77
and independence of specic paths are determined statistically. This model is likely
to be a close approximation of the multitude of factors that inuence the problem of
childhood obesity.
When a behavioral trial is guided by a conceptual model, it is reasonable to explore
a variety of outcomes in primary aims, including moderators and mediators of treat-
ment benet. These types of aims are consistent with the inclusive representation of
reality portrayed in the conceptual model. Exploratory studies based upon concep-
tual models are important and necessary for behavioral treatment development.
While they are highly informative in the early design phase, they are not well- suited
to the later conrmatory phase.
Individual
Psychosoclal
Factors
Conceptual Model
Hypothesized Pathway
Socio-
Environmental
Factors
Other Behaviors
Physical
Environment
Factors
Two-Components
IMMUTABLE
FACTORS
CONTEXTUAL
FACTORS
BEHAVIORAL/BIOLOGICAL
FACTORS
OBESITY
RISK
Activity-Related Factors
Dlet-Related
Factors
BMI
percentile/
Body
composition
Biological Factors
• Family
• Schools
• School
• Neighborhood
• Community
• Home
• Sleep
• Metabolism
• Pubertal status
• Blood chemistry
• Eating patterns
• Nutrient intake
• Substance use
• Weight control
• Activity/Sedentary patterns
• Activity/Sedentary amount
Planned for
(Weekly recall)
(Random home
visit)
(120 min/wk)
(≥7%)
Implementation
Targets
Behavioral Risk
Factor
Clinical
Endpoint
Treatment
Physical Activity
Most Days/Week
Physical
Activity
Weight
↓
↓
Presence of
triggers in home
--Reduce Eating
Triggers
--Plan Ahead for
Physical
Activity
• Peers
Demographic
Factors
Family
History &
Structure
Fig. 4.2 Conceptual model and hypothesized pathway for childhood obesity
Challenges forBehavioral Trials
https://t.me/medicina_free
78
Conrmatory clinical trials are guided by selective hypothesized pathways. An
example, derived from the conceptual model at the top of Fig.4.2, is presented on
the bottom of Fig.4.2. This pathway focuses on only two aspects of the home envi-
ronment and translates them into two treatment components. Each component has
an implementation target that quanties the “dose” of that component needed to
improve the behavioral risk factor target. It has further selected physical activity as
the behavioral risk factor target of the intervention which, if improved, is hypothe-
sized to reduce weight, the clinical endpoint. This selected subset of cause-and-
effect relations features unidirectional causality going from treatment to outcome.
The aim is less to explain inclusively and more to hypothesize a specic treatment
pathway which, if altered, will have a downstream effect on the primary clinical
endpoint of weight.
The hypothesized pathway has the characteristics of a scientic hypothesis. It is a
simple, relative to a complex, model. It “sticks its neck out” by targeting only two fac-
tors from the conceptual model and hypothesizing the pathway by which improve-
ments will translate into a benet on weight. It is falsiable because it is quantied. It
identies the level of implementation needed for each component to have an impact on
the behavioral risk factor, similar to identifying the dose of a drug needed to have a
biological effect. It identies a clinically signicant milestone for the level of improve-
ment needed on the behavioral risk factor (e.g., physical activity) to achieve improve-
ment on the clinical risk factor (e.g., weight). This clinically signicant milestone was
derived from clinical guidelines, based upon meta- analyses that identify the level of
improvement needed to achieve improvement in cardio-metabolic risk. Each step in the
pathway serves as a judge for the success of the behavioral treatment. Each step guides
the choice of essential measurements in the trial that will include measures for each
step in the pathway using the best, most objective, state-of-the-art instruments available.
The key difference between a conceptual model and a hypothesized pathway is the
difference between the aims of discovery and conrmation. Exploratory studies dis-
cover associations. Conrmatory trials test quantied models. Early in the process
of behavioral intervention development, exploration and discovery are at a premium.
Conceptual models can and should inform the design of a treatment. But as develop-
ment progresses, exploration shifts to conrmation, and conceptual models evolve
into rened hypothesized pathways. Conrmation is selective. It “sticks its neck out.”
Inadequate attention to the early phases of treatment development can lead to the
temptation to conduct behavioral trials which aim to simultaneously discover and
conrm. A key symptom of this is when a conrmatory trial features a wide variety
of primary outcomes, moderators, and mediators. This runs the risk of posing
scientic hypotheses that are not falsiable. Chance alone will account for some
positive associations.
Some exploration in a conrmatory clinical trial is a good use of resources. But it
should be articulated as such in exploratory aims. Combining discovery and conr-
mation into primary aims of a clinical trial is inconsistent with progressive transla-
4 Hypothesized Pathway andBias
https://t.me/medicina_free
79
tional science, inconsistent with the scientic method, risks undue participant
burden, and opens the door to bias, inconclusive results, and inability to replicate.
Discipline-Specic Approach toBias
Methods are specic to disciplines. Most applied behavioral scientists came of age
learning how to design experiments in the tradition of Campbell and Stanley [12].
This tradition places an emphasis on understanding the dynamics of a behavioral
treatment. When the primary aim of a behavioral trial is to understand treatment
dynamics, mechanisms, or moderators, as is characteristic of early Phase I and
some Phase II studies, a short-term follow-up is common, and the biases that endan-
ger validity are well-established within the behavioral sciences [12].
But when the aim is to determine efcacy on a chronic disease outcome or a surro-
gate for it, as is characteristic of Phase III efcacy trials and some Phase II efcacy
trials, unique biases emerge. When participants are followed over extended periods
of time, there is less experimental control. Participants can undergo a variety of non-
trial treatments and/or they can lose interest in continuing to participate. Biases in
Phase III clinical trials have been identied in a wide variety of publications, pri-
marily in medical and epidemiological journals.
When the treatment under investigation is behavioral in nature and the efcacy of
that treatment is based upon improvement in a chronic disease outcome or surro-
gate, a cross-disciplinary understanding of bias is needed. Biases identied in both
the traditions of experimental design and clinical trials must be avoided to pro-
mote rigor.
OVERCOMING THESE
CHALLENGES INA
BEHAVIORAL TRIAL
Guide Trial Design withaHypothesized Pathway
Figure 4.3 presents a template for a hypothesized pathway. The hypothesis to be
tested in a Phase III behavioral trial is that treatment (A) will have a clinically sig-
nicant
(
CS
)
benet on clinical outcome (D). Early phase studies of this treatment
show that it can be implemented by participants as designed (A→B) and that this
level of implementation can produce a clinically signicant change in the behav-
ioral or biomedical risk factor target of treatment (B→C). The link between A and
B demonstrates that participants can and will enact the treatment according to pro-
Overcoming These Challenges inaBehavioral Trial
https://t.me/medicina_free
80
tocol. The link between B and C is that this level of implementation will produce a
change in a behavioral risk factor that is strong enough to produce a change on a
clinical endpoint. Note the emphasis on clinically signicant
(CS)
change in both
the behavioral risk factor (C) and the clinical endpoint (D). These are categorical
milestones which, if achieved, will be linked to clinically meaningful reduction of
disease risk.
Figure 4.3 can be modied for a Phase II behavioral trial by eliminating (D). By so
doing, the primary outcome becomes the behavioral or biomedical risk factor in (C).
A hypothesized pathway, such as that shown in Fig.4.3, should be articulated early
in a program of research to guide progress at all subsequent stages. Feasibility stud-
ies can determine whether or not a treatment component can be implemented as
designed. Renement studies can determine whether a specic level of implementa-
tion is sufcient to improve a behavioral risk factor at a clinically signicant level.
Answers to these questions at early developmental phases provide the opportunity
to rene the behavioral treatment, as well as the hypothesized pathway, before mov-
ing to a conrmatory Phase II or Phase III behavioral trial.
The structure of a hypothesized pathway can be extended to explore mediators (C)
and biomedical mechanisms (E), as presented in Fig. 4.4. When mediators and
mechanisms are included in a hypothesized pathway, they are conrmatory, not
exploratory. Inclusion means that they must be improved for a treatment to have
benet on a clinical outcome. They become, in essence, a way to judge the success
of a treatment. Although an understanding of mediators and mechanisms increases
condence in a behavioral treatment, without certainty that they are part of the path-
way, it is risky to judge treatment success by change in them. Mechanisms of action
in drugs are often elusive, and their immense biological subtlety often presents itself
long after a drug has been on the market. Similarly, behavioral mediators and mech-
anisms are often elusive. Many behavioral treatments have produced clinically sig-
nicant benet without a clear understanding of the psychological or biomedical
mechanisms of action [26].
When there is uncertainty, mediators and mechanisms are better included as second-
ary or exploratory aims rather than part of a hypothesized pathway. Inclusion of a
mediator that is hypothesized not to change with treatment can be a powerful way
to support the discriminant validity of that treatment. In a progressive, translational
science model, mediators and mechanisms explored in the early phases of treatment
Fig. 4.3 Template for hypothesized pathway for a behavioral clinical trial
4 Hypothesized Pathway andBias
https://t.me/medicina_free
81
development could promote more certainty about their role in producing a benet
from a treatment, thereby justifying their incorporation into the hypothesized path-
way in a conrmatory Phase II or Phase III trial, as shown in Fig.4.4.
Interpret Trial Results withaHypothesized Pathway
A major benet of having a simple, quantied, a priori hypothesized pathway is that
it aids in the interpretation of results of a trial and decisions about appropriate next
steps. It is obvious when the result of a trial is positive or null. But without a hypoth-
esized pathway, it is less obvious why.
In a Phase III trial where the hypothesis is that a behavioral treatment will improve
a disease outcome, it is a failure of the treatment if milestones were not achieved in
the rst step (A to B in Fig.4.3) or in the second step (B to C in Fig.4.3). It is a
failure of the hypothesis when milestones were achieved in these rst and second
steps but they did not translate into achievement of the milestone in the third step (A
to D in Fig.4.3).
In a Phase II trial where the hypothesis is that a behavioral treatment will improve
a behavioral or biomedical risk factor, it is a failure of the treatment if milestones
were not achieved in the rst step (A to B in Fig.4.3) and a failure of the hypothesis
if milestones were achieved in the rst step, but they did not translate into benet on
the behavioral or biomedical risk factor at the second step (B to C in Fig.4.3).
In the event of a positive trial where benet on the primary outcome was achieved,
generalizability depends upon an ability to determine if it was by a known or
unknown pathway. If targets for every link in the hypothesized pathway were
achieved, the pathway of benet is known, and it would encourage replication by
the same or another group, an effectiveness trial in a real-world setting, and/or a
dismantling study to determine if the same effect could be produced more ef-
ciently. Alternatively, if targets were not achieved on one or more steps of the
hypothesized pathway, the interpretation would be that a positive effect was achieved
by an unknown pathway. In this situation, it would be of interest to rene the path-
way, based perhaps on qualitative studies in the participants, and determine if the
same effect could be produced in a new trial using a rened hypothesized pathway.
In the event of a null trial, it is incumbent on the investigator to determine why
thetrial failed. In a Phase III trial, a failure of the hypothesis would be the right
Fig. 4.4 Expanded template for hypothesized pathway for a behavioral clinical trial
Overcoming These Challenges inaBehavioral Trial
https://t.me/medicina_free
82
conclusion if a treatment was implemented according to protocol, the behavioral
risk factor target was reduced to a clinically signicant level, but the primary chronic
disease outcome was not improved. This might direct the investigator to abandon
this treatment approach, or this behavioral risk factor target, as a way to improve a
chronic disease. Alternatively, a failure of the treatment would be the right conclu-
sion if one or more links in the hypothesized pathway were not improved. Either the
treatment could not be implemented as designed or it was not strong enough to
achieve a clinically signicant improvement on the behavioral risk factor. A failure
of the treatment might direct the investigator to go back to the treatment develop-
ment phase to rene and optimize in places where weakness occurred. In past
behavioral clinical trials which produced null results, it is common to mistake a
failure to deliver the treatment as a failure of the trial hypothesis [27–29].
When the hypothesized pathway is evaluated against results for a comparator group,
it can provide insight into potential biases in the trial design or implementation.
Unexpected improvement in a comparator condition, particularly to levels that are
clinically signicant, could be a sign of treatment crossover or the choice of a con-
trol that was more active than intended.
Following are concrete examples of the usefulness of a hypothesized pathway to the
interpretation of results and decisions about next steps.
• In a null trial, the hypothesized pathway revealed that participants in the
comparator arm underwent clinically signicant improvement on the
behavioral risk factor. This may suggest that the comparator was an
active treatment. If the sample size in the trial was based upon the
assumption that the comparator group would not change from their base-
line level, the hypothesized effect size would be smaller than assumed,
and the trial will be underpowered to detect it. A future direction could be
to conduct a pilot study of several choices for comparators and determine
empirically the best one. Signicant change in the comparator group
could also suggest treatment contamination where participants sought
trial treatment on their own. Another future direction could be to repeat
the trial, keep track of all treatments participants received, and improve
the rigor of the trial operations by developing a culture characterized by
equipoise, the honest belief that the answer to the question of which arm
is better is unknown.
• In a null trial, the hypothesized pathway revealed that attendance at the treat-
ment sessions was only 30% when the goal was 80%. This was a failure to
implement the treatment. It would encourage qualitative analyses to obtain
the perspective of the participants on what could help to improve engage-
ment. If renement of the protocol bolstered attendance and a small proof-
of-concept study showed it produced clinically signicant benet, it would
justify replication of the trial rather than abandonment of the treatment.
• In a positive trial, the hypothesized pathway revealed that attendance at
thetreatment sessions was only 30% when the goal was 80%. This could
4 Hypothesized Pathway andBias
https://t.me/medicina_free
83
suggest that the implementation target was unnecessarily stringent and
only 30% of the treatment “dose” is needed. It could also suggest that an
alternative pathway may have produced benet. A replication of the same
trial but with a lower required dose of treatment in a small proof-of-con-
cept study could help to disentangle these possible explanations.
Develop Cross-Disciplinary Sophistication inCommon
Trial Biases
Before making the judgment that a hypothesis has been tested and a conclusive result
obtained, it is essential to rule out potential biases that may provide alternative expla-
nations for results and thus compromise the trial. While it is clearly preferable to
anticipate and minimize bias before undertaking a trial, identication of unexpected
biases that may be emerging during the course of a trial, or have occurred in a com-
pleted trial, aids in its its interpretation. There are ve general categories of bias.
1. The Passage of Time. Participants would have improved or worsened due to the
passage of time, without any treatment. These biases provide the essential justi-
cation for use of a randomized design since participants in all arms will experi-
ence these extraneous factors similarly.
2. Errors in Statistical Inference. Violations of assumptions underlying statistical
procedures can spuriously enhance or undercut an inference of covariation
between a treatment and an outcome.
3. Preferences. When participants, staff, providers, and/or investigators view treat-
ment arms as “favored” and “unfavored,” reactivity to that value-laden label
inuences behavior. These biases provide the justication for extending the blind
in as many ways, and to as many people, as possible.
4. Unintended Treatments. Part of active treatment could include other, unintended
factors that covary with a dened treatment. This could spuriously potentiate the
treatment effect but undercut the ability to replicate it. It is important to distin-
guish treatment effects from nuisance variables and clarify this before beginning
a trial.
5. Dissimilar Treatments. Treatments that look different and have different opera-
tional demands can inadvertently promote differential trial procedures that affect
the quality of data collection.
Table 4.1 presents a list of specic biases that have been identied within each of
these ve categories. All have presented problems for Phase II or Phase III behav-
ioral clinical trials in the past. It is a synthesis of biases identied in experimental
research in the behavioral and social sciences [12], clinical trials in medicine [30],
and epidemiologic methods [11]. It includes a column labeled “Prevention” which
presents a general suggestion for how to minimize the risk of the particular bias in
trial design, implementation, or interpretation. Suggestions in bold apply to all
biases in the category; other suggestions are type-specic.
Overcoming These Challenges in a Behavioral Trial
https://t.me/medicina_free
84
Table 4.1 Selected biases common in behavioral clinical trials
SOURCE
OF BIAS SPECIFIC BIAS EXPLANATION PREVENTION
Passage of
time
Since the outcome could
have occurred over time
without treatment, it is a
mistake to infer treatment
caused it
– Randomized design
Spontaneous
remission/
Maturation
Natural changes occurring in
absence of treatment (e.g.,
older, sicker, spontaneous
remission of disease)
Historical events/
Secular trends
Events occurring during
follow-up that affect outcome
(e.g., policy changes,
advertising campaigns,
secular trends in disease)
– Parallel cohort designs
Regression Participants selected because
of extremeness tend to be less
extreme upon retest (e.g.,
high blood pressure, elevated
stress, overweight)
– Repeated baseline
eligibility assessments
Testing Familiarity with a test due to
repeated testing over time is
confounded with treatment
effects (e.g., cognitive
function)
– Statistical adjustment
for testing effects
– Parallel forms of tests
Instrumentation Change in measuring
instrument over time is
confounded with treatment
effects (e.g., adjudicators
become more procient)
– Ongoing calibration of
measuring instruments
and outcomes assessors
Statistical
inference
The inference of covariation
between a treatment and
outcome is weakened by
violation of statistical
assumptions
Low power Low probability of rejecting
the null hypothesis when the
alternative hypothesis is true
and thus a high probability of
Type II error
– Increase sample size
– Increase strength of
treatment
– Decrease diversity of
participants/settings
– Less stringent
comparator
Violation of
statistical
assumptions
Data may not meet the
parametric statistical test
assumptions of normality of
distribution or independence
of observations
– Non-parametric tests
for non- normal
distributions
– Cluster randomized
designs for correlated
observations
(continued)
4 Hypothesized Pathway andBias
https://t.me/medicina_free
85
Table 4.1 (continued)
SOURCE
OF BIAS SPECIFIC BIAS EXPLANATION PREVENTION
Fishing/Data
snooping/“Cherry
picking”
Multiple statistical tests
conducted on multiple
outcomes increase the
probability of a Type I false
positive error
– Single primary outcome
– Single behavioral risk
factor target
– Statistical adjustment of
alpha
Unreliable
measures
Low reliability biases toward
the null hypothesis
– Identify valid and reliable
outcome measures before
starting a trial
Spectrum bias/
Restriction of
range/Ceiling
effect
Restricted range of disease
outcome or behavioral risk
factor promotes oor/ceiling
effects and biases toward the
null hypothesis
– Include diverse
populations and tailor
treatment to enhance
efcacy within
subgroups
Preferences Differential expectations of
benet undercut
equivalence created by
randomization
– Foster a culture of
equipoise throughout
trial
– Extend the blind to as
many players as
possible
– Blind to hypotheses
but not to trial aims
Co-interventions/
Compensatory
treatment
Providers, families offer more/
less non-trial treatment based
upon expectations of benet
from the trial arm to which
participant was randomized
– Assess all non-trial
treatments undergone
by participants
Treatment
contamination/
Underdog/John
Henry
Participants randomized to
comparator seek trial
treatment on their own
because they failed to get the
desired treatment
– Assess all non-trial
treatments undergone
by participants
Demoralization Participants randomized to
comparator lose interest,
compromising adherence,
outcomes assessment, and
running the risk of differential
dropout
– Monitor attendance and
drop-out rates closely in
both arms
– Provide attractive
compensation for
completing the primary
outcome at the end of
the trial
Ascertainment
bias
Outcomes assessors become
aware of the arm to which the
participant is randomized and
judge outcomes differentially
based upon their arm-specic
expectations of benet
– Blind outcomes
assessors to the
treatment arm to which
participant is
randomized
– Instruct participants not
to engage in discussions
with staff about
treatment activities
(continued)
Overcoming These Challenges in a Behavioral Tria
https://t.me/medicina_free
Соседние файлы в папке Библиотека им академика М.И. Перельмана