Добавил:
Sekretar
kiopkiopkiop18@yandex.ru
t.me/Prokururor I Вовсе не секретарь, но почту проверяю
Опубликованный материал нарушает ваши авторские права? Сообщите нам.
Вуз:
Предмет:
Файл:Ординатура / Хирургия / Библиотека им академика М.И. Перельмана / Книга_2946_Библиотеки_им_академика_М_И_Перельмана
.pdf
Markov models 255
stationarity cannot be assumed for p
14
. Instead, the value for p
14
should match the actuarial hazard rates described in
Chapter11. Those actuarial hazard rates grow as the patient ages, which means the transition probability p
14
should
change over time. Changing values for p
14
contradicts the stationarity assumption.
Possible changes in p
14
matter less if the time spent in the Disease- free health state is short. In turn, the time spent in
the Disease- free health state will be short if transitions to either the Local recurrence or Distant recurrence health
states are highly likely. In other words, if the transition probabilities p
12
or p
13
are large, the patient’s health state is
Disease- free for only a few time intervals. In this case, the changes in p
14
will be insignificant during the short time the
patient is free of disease.
In general, the validity of the stationarity assumption decreases as the time spent in health states increases. Therefore,
stationary Markov models for medical conditions in which the patient has a high likelihood of remaining disease free
are problematic. The mortality rates for a disease- free patient increase as the patient ages. The longer the patient
remains disease free, the more the mortality rate faced by the patient departs from any assumed constant value.
Disease
free
(1)
Dead
(4)
Local
recurrence
(2)
Distant
recurrence
(3)
Mortality rate with
distant recurrence
Mortality rate without
breast cancer recurrence
p
11
p
33
p
12
p
13
p
23
p
24
p
44
p
34
p
14
p
22
Figure12.6 Copy of the Markov model shown in Figure12.3.
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0102030405060708090100 11
0120
P [Alive at x months]
Months after diagnosis of recurrence, x
Exponential survival model
Kaplan–Meier survival model
Figure12.7 Comparison of Kaplan–Meier model of survival and exponential survival model for patients with a breast cancer distant recurrence.
Adapted from Stokes etal. (2008).
https://t.me/medicina_free
256 Medical decision making
Three different approaches are used to address problems with the stationarity assumption. Perhaps the simplest
approach is to use average values for transition probabilities that change with time.
For example, the breast cancer recurrence Markov model shown in Figure12.6 includes a Disease- free health state.
Under the stationarity assumption, as long as the patient remains in this health state, they face an exponential survival
model because the mortality rate (p
14
) is constant. According to the life table analysis discussed in Chapter11, the life
expectancy is 12.4 years for a 75-
year- old woman who is free of disease. Therefore, with an exponential survival
model the mortality rate should result in a life expectancy of 12.4 years for a 75-
year- old woman who remains free of
disease. Recall that with an exponential survival model the mortality rate is one over the life expectancy, or for a
75-
year- old woman:
rtality rate
years
/year= =
1
12 4
0 0804
.
.
This suggests an average value of 0.0804/year for the mortality rate represented by p
14
. Later, when we discuss the
estimation of transition probabilities we will find this first approach to dealing with the stationarity assumption to
beuseful.
The second more complicated approach to addressing problems with the stationarity assumption is to use additional
health states to represent changes in transition probabilities as the patient ages. Figure 12.8 shows a portion of a
Markov model that demonstrates how this would be done with the breast cancer problem we have been discussing.
Once again, the breast cancer recurrence Markov model, shown in Figure12.6, includes a Disease- free health state.
Mortality rates for the patient in this health state are represented by the transition probability p
14
, which must be con-
stant under the stationarity assumption. Therefore, if the patient is a 75-
year- old woman at the start of the analysis, the
logical value for p
14
would be annual mortality rate for 75- year- old woman. According to the actuarial survival model
discussed in Chapter11, the mortality rate for a 75-
year- old woman is 0.0249/year. However, the mortality rate for
women increases to 0.0743/year by the time the patient reaches the age of 85 years. In other words, using the morality
rate for a 75- year- old woman as the value for p
14
greatly underestimates the probability the patient will die if she
survives to the age of 85 years.
Disease-free
age 75
(1)
p
15
p
14
p
12
p
13
p
54
p
52
p
22
p
24
p
44
p
23
p
34
p
33
p
53
p
62
p
63
p
64
p
73
p
72
p
74
Disease-free
age 78
(7)
p
56
p
67
Dead
(4)
Uncured
local
recurrence
(2)
Uncured
distant
recurrence
(3)
Disease-free
age 76
(5)
Disease-free
age 77
(6)
Figure12.8 Breast cancer Markov model that accounts for changes in actuarial mortality rates.
https://t.me/medicina_free
Markov models 257
The Markov model shown in Figure12.8 avoids this underestimation by adding age- specific disease- free health
states. Starting in the Disease-
free age 75 health state, after 1 year, the patient changes to the Disease- free age 76 health
state, assuming the patient neither dies nor experiences a breast cancer recurrence. During the first year, the probability
of advancing 1 year is p
15
in Figure12.8. Since the transition probabilities add up to one for any health state
p ppp pppp
12 13 14 15 15 12 13 14
11 or
The transition probability p
14
equals the annual mortality rate for a 75- year- old woman. Similarly, the transition prob-
ability p
54
equals the annual mortality rate for a 76- year- old woman, and so forth. The Markov model would be
extended to include age-
specific disease- free health states for all of the ages the patient might expect to reach.
The Markov model shown in Figure12.8 also can represent how the probabilities for the occurrence of local or dis-
tance recurrences change with the passing of time. Often, the probability of a recurrence decreases as a patient contin-
ues to be disease free. The transition probabilities originating from the corresponding age-
specific disease- free health
state would represent this change in the uncertainty faced by the patient. Therefore, p
52
would be less than p
12
and p
53
would be less than p
13
.
This second approach to dealing with the stationarity assumption fully avoids any distortions caused by how transi-
tion probabilities change overtime. However, this approach complicates analyzing the resulting Markov model because
of the large number of required health states.
Finally, the third approach is to use a stationary Markov model that allows for changes in the transition probabilities.
This approaches fundamentally changes how Markov models are analyzed. However, this approach is often used in
the analysis of complex policy questions, as will be discussed later.
12.2.5 Acyclic graph assumption
Some readers might notice that all of the Markov models described so far in this chapter are acyclic. By “acyclic” we
mean that the possible transitions in the model are such that once a health state has been left, there is no sequence of
transitions that return back to that same health state.
In case of cancer recurrence, the lack of a return to the Disease- free health state in Figure12.6 would appear to
exclude the possibility of successful salvage treatments. Of course, salvage treatment often can return the patient to a
disease- free condition after a recurrence. An arc from the corresponding recurrence state to the Disease- free health
state would represent the possibility of a return to the disease- free condition.
One answer to this apparent gap in the Markov models we have discussed is that the definitions of the Local recur-
rence and Distant recurrence health states contain the possibility of a salvage treatment. Presumably, a patient with a
recurrence would undergo further treatment. The possible effects of additional treatments would be reflected in the
mortality rates for the corresponding health states. In other words, the Markov models that have been discussed imply
the existence of salvage treatments in the definition of the corresponding health state.
On the other hand, the Markov model shown in Figure12.9 makes the existence of salvage treatments explicit in the
case of a local recurrence.
1
This Markov model has the same health states as the four- state models that have been
discussed. However, notice that an arc has been added showing the possibility of a transition from the Local
1
In order to simplify the Markov model diagrams, the discussion will focus only on the Local recurrence health state.
Methods forrepresenting nonstationary transition probabilities
1. Use an average value to represent the range of values for a transition probability that changes over time.
2. Add health states that represent the different values for a transition probability that changes over time.
3. Use a nonstationary Markov model.
Acyclic
A Markov model is acyclic if, for every health state, there is no possible sequence of transitions that return to
health state once it has been left.
https://t.me/medicina_free
258 Medical decision making
recurrence health state back to the Disease- free health state. The corresponding transition probability p
21
quantifies the
likelihood that salvage treatment will be successful.
However, what does it mean to return to the Disease- free health state? Most likely, the chance of another local recur-
rence in the future has changed. This means the transition probability p
12
is not the same, which contradicts the Markov
independence assumption. Moreover, for many patients recovering from a recurrence is not the same as remaining
disease free because of the psychological trauma of having breast cancer return.
Figure12.10 shows an acyclic Markov model that represents salvage treatment for a local recurrence. In this alterna-
tive model, the Local recurrence health state has been replaced by a Local recurrence treatment health state. This new
Disease
free
(1)
Dead
(4)
Local
recurrence
(2)
Distant
recurrence
(3)
p
11
p
33
p
12
p
21
p
13
p
23
p
24
p
44
p
34
p
14
p
22
Probability salvage
treatment is successful
for local recurrence
Figure12.9 Cyclic version of four- state Markov model for breast cancer survival with explicit representation of salvage treatment for local
recurrence.
Dead
(4)
Local
unmanaged
(6)
Distant
recurrence
(3)
Local
recurrence
managed
(5)
p
11
p
14
p
13
p
44
p
54
p
53
p
56
p
34
p
25
p
33
p
55
p
66
p
64
p
63
p
12
p
22
p
26
p
24
Disease
free
(1)
Local
recurrence
treatment
(2)
Figure12.10 Markov model representing possible recovery from a local recurrence of breast cancer.
https://t.me/medicina_free
Markov models 259
health state represents the salvage treatment that is started when a local recurrence is found. The patient remains in
this health state for one or more time intervals.
Assuming the patient does not die during salvage treatment, they face two possibilities. One possibility is the patient
moves into the Local recurrence unmanaged health state if the salvage treatment fails. In this health state, the local
recurrence continues, and the patient faces the increased likelihood of progressing to a distant recurrence. The other
possibility is the patient moves into the Local recurrence managed health state if the salvage treatment is successful.
This health state is much like the Disease-
free health state, except that the patient continues to experience whatever
residual anxiety they feel because of the recurrence. Also, in the Local recurrence managed health state, there is the
possibility of proceeding to a distant recurrence. However, that possibility is less than it would be if the salvage treat-
ment had failed. That is p
53
would be less than p
63
.
The uncertainty about the time spent enduring salvage treatment is represented by the survival probability p
22
. If the
salvage treatment lasts exactly 1 time interval p
22
would be zero. Larger values for p
22
would represent longer times for
the salvage treatment. The relative size of the transition probabilities p
25
and p
26
represents the relative likelihood that
salvage treatment will succeed.
The Markov model in Figure12.10 also represents the possibility that salvage treatment, which initially appears to
be successful, can later be followed by a subsequent appearance of a local recurrence. The transition probability p
56
measure the likelihood of that possibility.
The reason for discussing the Markov model in Figure12.10 is to demonstrate how these models can be extended
to capture the details in a medical decision without reverting to models that are not acyclic. Avoiding returns to
health states avoids the medical contradictions that have been discussed. Acyclic Markov models also are easier
toanalyze.
12.3 Determining transition probabilities
The previous section shows that Markov models provide structures that represent the dynamics, as well as the uncer-
tainty, for the outcomes that can follow a decision. Those structures are collections of health states that are linked by
transitions representing changes in the patient’s condition over time. The probabilities for those transitions quantify
the uncertainty in how an outcome will unfold.
This section shows how to determine the transition probabilities used in a Markov model. The process that will be
described is complicated in places and may be challenging for some readers. Moreover, understanding this process is
not needed to appreciate the role for Markov models in analyzing medical decisions. Therefore, this section can be
skipped. However, transition probabilities are essential to the definition of a Markov model. Readers involved in the
use of Markov models should understand how values are determined for transition probabilities.
A transition probability is, of course, a probability. Any of the methods discussed elsewhere in this book for assign-
ing a value to a probability could be used. This includes subjective probability assessment. However, this section
focuses on empirical methods for determining the value of a transition probability.
It should be noted that the work involved in determining the transition probabilities typically requires more effort
than selecting the states and transitions that define a Markov model. The work required to determine the transition
probability values usually must bridge the gap between the medical concepts embodied in a Markov model and what
is available from well- documented studies. The paucity of credible studies can force the restructuring of a model. This
means the processes that will be described in this section are a critical part of Markovian analysis.
12.3.1 Markov model used toillustrate how transition probabilities are determined
The five- state Markov model, shown in Figure12.11, will be used to illustrate how transition probabilities are deter-
mined. This model represents the survival of a 75-
year- old woman who has been diagnosed with a breast cancer recur-
rence. This patient’s survival can follow two possible paths. One path is successful salvage treatment that completely
eliminates the breast cancer that has reappeared. In this case, the patient’s survival matches that of any similarly aged
woman. This possibility is represented by the Successful treatment health state in Figure12.11.
The other possibility is the breast cancer recurrence does not respond to treatment. In this case, the patient will still
face the same survival risks that are faced by anyone of their age. However, unsuccessful treatment also means the
patient faces an additional risk of breast cancer that has reappeared. The amount of that additional risk depends on the
stage of the recurrence. The possibility of unsuccessful salvage treatment is represented in Figure12.11 by the health
states Local recurrence and Distant recurrence.
https://t.me/medicina_free
260 Medical decision making
Notice that the Markov model shows a transition from the Local recurrence health state to the Distant recurrence
health state. This transition represents the increased risk that comes with a local recurrence. A patient with a local
recurrence that cannot be controlled faces a different survival model than a patient without a recurrence. However, the
increased risk faced by this patient is not because of the local recurrence itself. A tumor that exists only at the site of the
original breast cancer usually does not become life threatening until it spreads to a vital organ, in other words, until it
metastasizes and becomes a distant recurrence. The probability p
23
measures the speed of that transition.
Finally, notice that the Markov model in Figure12.11 does not show a survival transition for the initial Breast cancer
recurrence health state. This means the patient remains in the initial health state for exactly 1 time interval, which will
be 1month in this example. The time spent in the Breast cancer recurrence health state represents the duration of the
salvage treatment. In order to simplify the discussion this example will assume treatment lasts exactly 1month.
12.3.2 Determining mortality rates
We will start by determining values for the mortality rates in the Markov model. These are the transition probabilities
for the arcs leading to the Dead health state (see Figure12.12). As already noted, the mortality rate for the Successful
treatment health state is the mortality rate for a 75-
year- old woman. If we were constructing a nonstationary Markov
model this transition probability (p
54
) would change, as the patient aged, to match the actuarial hazard rates for a
75-
year- old woman. However, we will focus on stationary Markov models and use an average value for p
54
.
The life expectancy for a 75-
year- old woman is 12.4 years or 149months. This means the value for p
54
is the constant
mortality rate that results in a life expectancy of 149 months. The discussion of exponential survival models in
Chapter11 noted that when the mortality rate is constant, the patient’s life expectancy is one over the patient’s mortal-
ity rate. In a stationary Markov model, all of the mortality rates are constant. Therefore, the patient’s life expectancy of
149months means they face the following mortality rate in the Successful treatment health state:
p
54
1
149
0 0067= =
months
/month.
Breast
cancer
recurrence
(1)
Dead
(4)
Local
recurrence
(2)
Successful
treatment
(5)
Distant
recurrence
(3)
p
13
p
23
p
22
p
24
p
34
p
14
p
54
p
55
p
15
p
33
p
44
p
12
Figure12.11 Markov model of breast cancer recurrence used to demonstrate how transition probabilities are determined.
Mortality rates inMarkov models
In a Markov model, a mortality rate is the transition probability for an arc connecting a health state to a trapping
state representing death.
https://t.me/medicina_free
Markov models 261
It follows that
pp
55 54
110 0067 0 9933 ../month /month
Next, we will consider the mortality rate for the Distant recurrence health state (p
34
). This transition probability
measures the likelihood of death during a 1- month time interval if the patient has metastatic breast cancer that has not
been treated successfully.
The survival curve in Figure12.13 can be used to determine the risk of dying, each month, faced by a patient with
metastatic cancer. We first saw this survival curve in the discussion of the two-
part survival model back in Chapter11.
The patients represented in this curve are from a group of 622 women with distant recurrence of breast cancer
(Stokesetal.,2008).
Breast
cancer
recurrence
(1)
Dead
(4)
Local
recurrence
(2)
Successful
treatment
(5)
Distant
recurrence
(3)
p
13
p
23
p
22
p
24
p
34
p
14
p
54
p
55
p
15
p
33
p
44
p
12
Figure12.12 Transition probabilities representing the mortality rates in the Markov model.
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1. 0
01224364
860
P[Alive at x months]
Months after diagnosis of recurrence, x
12 months
0.3331
Figure12.13 Kaplan–Meier survival model for patients diagnosed with a distant recurrence of breast cancer. Adapted from Stokes etal. (2008).
https://t.me/medicina_free
262 Medical decision making
The top row in Table12.1 contains the values for the survival probabilities, shown in Figure12.13, for various follow-
up times. For example, 12months after the first detection of the recurrence, the probability that a patient was alive is
0.3331. In order to determine the transition probability p
34
we must determine the corresponding 1- month survival
probability. Let p denote that 1-
month survival probability. Then p
2
would be the probability the patient survives
2months, p
3
would be the probability the patient survives 3months, and so forth. Therefore, the probability the patient
survives for a year is p
12
, which we know to be 0.3331. That is
p
12
0 3331= .
Using a little high school algebra, the probability the patient survives 1month is then
p = =0 3331 0 9125
112
..
/
The second row in Table12.1 repeats this calculation for several follow- up periods. Note the agreement between the
implied 1-
month survival probabilities. The average value for these 1- month survival probabilities is 0.9260/month.
What have we computed so far? The value 0.9260 approximates the average probability that a patient will survive
each month while they have a distant recurrence of breast cancer. In other words, one minus 0.9260/month, or
0.0740/month, is the probability of death during a 1-
month period for the patient in the Distant recurrence
healthstate. This means,
p
34
0 0740= ./month
Therefore, we have determined the mortality rates for two of the health states: Successful treatment and Distant
recurrence. It turns out that we already determined the mortality rate for the Local recurrence health state.
Recall the observation made earlier that the increased risk faced with an uncontrolled local recurrence is not because
of the local recurrence itself. A recurrence usually is not life threatening as long as it remains at the original site in
thepatient’s breast. Of course, a local recurrence is not a pleasant experience. Therefore, the quality of life is less for the
Local recurrence health state than it is for the Successful treatment health state. However, both health states have
roughly the same mortality rates. This means that
pp
24 54
0 0067= = ./month
The remaining mortality rate that must be determined is for the initial Breast cancer recurrence health state. Recall that
this health state represents the experience of a patient who will undergo treatment because breast cancer has recurred.
The transition probability p
14
represents the corresponding mortality rate. In general, recall that a transition probability
measures the likelihood of a transition during the time unit used to define the model, which is 1month in this example.
Therefore, the transition probability p
14
measures the likelihood of death during the 1month period following the detec-
tion of the cancer recurrence. Presumably, the patient will undergo treatment starting at the beginning of that month.
Breast cancer treatments are painful and can be disfiguring; however, usually, they are not dangerous. Therefore, we
will ignore the risk of death from the procedure itself. However, two other issues affect the patient’s chance of dying
during the initial month. The first issue is the extent of the breast cancer recurrence that has been detected. As we have
already discussed, the cancer itself does not threaten the patient’s life if the recurrence is local. However, with a metas-
tasis, the patient does face an increased mortality rate from the cancer, since it has spread to what may be a vital organ.
The second issue affecting the chance of death for the patient in the initial Breast cancer recurrence health state is the
success of the salvage treatment. If the treatment is successful, the patient then faces the mortality rate for a typical
75- year- old woman during the first month because that is what we mean when we say the treatment is successful.
If the treatment is not successful, the mortality rate for the first month depends on the extent of the cancer recurrence.
With a local recurrence, the patient still faces the mortality rate for a typical 75- year- old woman during the firstmonth.
Table12.1 Monthly mortality rates fordistant recurrence ofbreast cancer.
Months since first detection of distant recurrence
12 24 36 48 60 Average
Observed survival probability 0.3331 0.1477 0.0743 0.0349 0.0142
Implied 1-
month survival probability 0.9125 0.9234 0.9303 0.9325 0.9316 0.9260
https://t.me/medicina_free
Markov models 263
That local recurrence will eventually spread and become a distant recurrence, which will increase the mortality rate.
However, that mortality rate increase will occur during a later month. On the other hand, unsuccessful treatment of a
distant recurrence means the patient faces elevated risk during that first month.
Therefore, during the first month, the patient will face the mortality rate for a 75-
year- old woman (0.0067/month) if
either (1) the recurrence is local or (2) the recurrence is distant, but the treatment is successful. Otherwise, the patient
will face the mortality rate associated with a distant recurrence, which we found to be 0.0740/month when we deter-
mined the value for p
34
.
Combining these observations into a single expression for p
14
benefits from the use of a more compact notation.
Denote the probabilities for a local and distant recurrence as follows:
Local
and
Distant
Similarly, the following notation will denote the probability of success given the type of recurrence:
Local
and
Distan
t
As noted earlier, the patient’s mortality will be 0.0067/month during the first month if either (1) the recurrence is
local or (2) the recurrence is distant but the treatment is successful. Using the notation we just defined
Local
Similarly,
DistantD
iistant
Therefore,
ppMortality rate /month
LocalDistant Distant
0 0067.
Similarly, the patient’s mortality will be 0.0740/month during the first month if they have a distant recurrence and
unsuccessful treatment. That is
p
Mortality rate /month
DistantDistant
0 0740 1.
Combining these two results leads to the following expression for the patient’s mortality rate in the Breast cancer
recurrence health state:
pp pp
14
0 0067 1
LocalDistant DistantDistant Di
/month
.
sstant
/month
0 0740.
The probabilities used in this expression can be determined from the study that provided the survival curve for
distant recurrences, shown in Figure12.13. That same study also provided survival data for local recurrences. In addi-
tion to the 622 patients with distant recurrences, there were 958 patients with local recurrences. Since these patients
were found in a population of unselected breast cancer cases, the patient counts of 958 and 622 represent the relative
proportion of recurrences that are local or distant, respectively. That is
roportion of recurrences that are local
958
958 622
0
6063
.
roportion of recurrences that are distant
622
958 622
039
.3
37
https://t.me/medicina_free
264 Medical decision making
Therefore,
p
Local
=0.6063and p
Distant
=0.3937
In order to determine the probability that treatment will be successful, recall how the two-
part survival model was
developed in the previous chapter for a patient with a distant recurrence of breast cancer. The number of expected
deaths were subtracted from the deaths observed in the 622 patients during follow- up. This calculation implied that
550 patients, or 87.57%, died from cancer. Dying from cancer means treatment was not successful, implying
Distant
10.. .8757 0 1243
A similar analysis of the 958 patients with local disease implies that
Local
10545.9904541 .
We now have the values needed to compute the mortality rate for the patient during the first time interval.
pp pp
14
0 0067 1
LocalDistant DistantDistant Di
/month
.
sstant
/month
0 0740.
0 6063 0 3937 0 1243 0 0067 0 3937 101243 00.... .../month 7740/month
This completes the determination of the mortality rates for the Markov model.
12.3.3 Determining probability fortransitions between stages ofrecurrence
Next, we will determine the probability of a transition between the Local recurrence health state and the Distant recur-
rence health state (p
23
). This transition represents the inevitable spread of a local recurrence when treatment has not
been successful. As has already been emphasized, the possibility of this transition is how a local recurrence reduces the
patient’s survival.
Figure12.13 showed the survival model for a group of patients with breast cancer recurrence that had spread to
distant locations by the time of diagnosis. Figure12.14 shows the corresponding survival model for patients from the
same study with a local recurrence of breast cancer. In other words, the newly diagnosed recurrences in this second
group of patients were near the sites of the original tumors in the patients.
Local recurrence
Distant recurrence
12 months
0.6889
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1. 0
01224364
860
P[Alive at x months]
Months after diagnosis of recurrence, x
Figure12.14 Kaplan–Meier survival model for patients diagnosed with a local recurrence of breast cancer. The corresponding survival curve for
local recurrence also is shown. Adapted from Stokes etal. (2008).
https://t.me/medicina_free
Соседние файлы в папке Библиотека им академика М.И. Перельмана