Ординатура / Офтальмология / Английские материалы / Study Design and Statistical Analysis a practical guide for clinicians_Katz _2006

149

Definition

Bayes’ theorem is a method for determining the

probability of an event based on the pretest probability of the event and the information added by a test.

The pretest probability is the likelihood of the condition prior to considering new evidence.

The posttest probability is the likelihood of a condition given the pretest probability of the condition and the information added by the test.

Bayes’ theorem

test.138 It is the theoretical basis for predictive models (in contrast to explanatory models which are based on inferential statistics).

Although Bayes’ theorem sounds complicated, we incorporate Bayes’ theorem in our everyday clinical decision-making. For example, imagine that two patients roll into the emergency department: Ms. Jones and Mr. Smith. They have identical complaints of substernal chest pain. Ms. Jones is a 34-year-old woman, non-smoker, with no prior medical history. Mr. Smith is a 55-year-old man with hypertension, diabetes, elevated cholesterol, and a 25-pack year history of smoking. As the patients have acute chest pain, the nurse immediately obtains an electrocardiogram (EKG) on both. As she hands them to you she rattles off a one-sentence description of the patients.

Now, before you even look at the EKGs, you have a different sense of these two patients. Although it is not impossible that Ms. Jones is having a myocardial infarction, the likelihood that a pre-menopausal non-smoking woman has heart disease is very low. In Bayesian terms, she has a low pretest probability of coronary artery disease. The pretest probability is the likelihood of a condition (e.g., myocardial infarction) prior to considering the new evidence (e.g., EKG).

In contrast, Mr. Smith has a high pretest probability of having a myocardial infarction because he has four risk factors (male, hypertension, diabetes, elevated cholesterol). To know the true pretest probability for both Ms. Jones and Mr. Smith, you would have to conduct a study looking at the proportion of patients presenting to an emergency department with risk factor profiles similar to theirs who turn out to have a myocardial infarction. However, even without knowing the exact pretest probability, we can say with certainty that it is much higher for Mr. Smith than Ms. Jones.

A diagnostic test, in this case the EKG, gives you a new probability of outcome, the posttest probability, which is conditional on the pretest probability and the new information, as shown in Figure 8.2.

For example, the low pretest probability for myocardial infarction for Ms. Jones would become higher if she had 4 mm of ST elevation in leads V1–V4. The high pretest probability for Mr. Smith would become lower if he had a normal EKG.

How different the posttest probability is from the pretest probability depends on the likelihood ratio of the test. The likelihood ratio for a dichotomous test is

To use multiple likelihood ratios to determine the posttest odds of an outcome, the likelihood ratios must be based on test results that are independent of one another.

Seeing that the child’s eardrum is cloudy changes the probability that the ear is infected from 0.20 to 0.89 and would likely result in prescribing antibiotics for the child.

An important feature of likelihood ratios is that they can be used to calculate the posterior probability of a disease based on the results of multiple diagnostic tests by multiplying the likelihood ratios together:

This is potentially useful because in most clinical situations we have more than one relevant test result and wish to use each piece of information to determine a patient’s diagnosis or prognosis. However, to incorporate more than one likelihood ratio in this way the likelihood ratios have to be based on test results that are independent of one another.

Unfortunately, many test results have not been demonstrated to be independent of one another. For example, it is not known whether the test results underlying

Tip

The standard for what determines an outcome should be specified prior to the start of the study.

Definition

The gold standard is the most rigorous standard available for assessing the characteristics of new tests.

When you are performing a predictive study, you need to define what standard you will use for deciding that a subject has a particular disease or outcome. The standard should be chosen prior to the start of the study to avoid biasing your choice by using information you have learned in the data collection process.

Ideally, you should test the characteristics of a diagnostic or prognostic test against the most rigorous standard available. This is referred to as the gold standard.

However, the most rigorous standard may not be feasible because of safety concerns, acceptability to subjects, and cost. Indeed, different investigators in the same field may choose different standards based on their sense of what is feasible.

For example, two studies of the ability of helical computed tomography (CT) to diagnose pulmonary embolisms (clots in the lung) used different standards. Qanadli and colleagues used the gold standard: pulmonary arteriography.142 In contrast van Strijen and colleagues used a clinical standard.143 Instead of subjecting patients to angiography, an invasive test that can result in serious adverse reactions, they performed compression ultrasonography to check for blood clots in the legs of subjects who had normal helical CT scans (identifying a blood clot in the leg makes a pulmonary embolism more likely and is a reason for initiating anticoagulation regardless of whether the patient has a pulmonary embolism). In addition, they followed subjects clinically for development of respiratory symptoms consistent with pulmonary embolism.

If you intend to use the clinical course of your patients rather than the gold standard to determine the clinical outcome it is critical that you:

1.Set the criteria of what will determine an outcome a priori (prior to the start of the study).

2.Create an independent committee to review the clinical course of the subjects.

3.Maintain close contact with your subjects.

In the case of the study by van Strijen and colleagues, there was an independent committee that reviewed all clinical data and no subjects were lost to follow-up.

142Qanadli, S.D., Hajjam, M.E., Mesurolle, B., et al. Pulmonary embolism detection: prospective evaluation of dual-section helical CT versus selective pulmonary arteriography in 157 patients. Radiology 2000; 217: 447–55.

143Van Strijen, M.J.L., de Monye, W., Schiereck, J., et al. Single-detector helical computed tomography as the primary diagnostic test in suspected pulmonary embolism: a multicenter clinical management study of 510 patients. Ann. Intern. Med. 2003; 138: 307–14.

Tip

To avoid spectrum bias test your predictive model in a setting similar to the one in which the model will be used.

Predictive models should be tested in samples that are similar to the ones that the model will be used in. In particular, the prevalence and severity of disease, along with the prevalence and severity of diseases that mimic the disease under study, will all affect the performance of the model.

If you choose a sample where persons are likely to have advanced disease, the sensitivity of a test will likely be higher than it will in samples where the disease is at an earlier stage. Conversely, if you select a sample of very healthy controls, the specificity of the model will be higher than in populations where there is a high prevalence of other diseases that could mimic the disease you are studying. This is referred to as spectrum bias. Spectrum bias occurs when predictive models are tested on samples that are not representative of the person for whom the model will be used.144

With novel associations, be especially rigorous in assessing the possibility that the association is spurious.

Associations that have been documented in prior studies are more likely to be true than completely novel associations. In Bayesian terms, when prior studies have shown an association to be present, there is a higher prestudy (pretest) probability that the finding is true.

Of course, someone has to be the first to document a true association, and science would not progress much if we all ignored new associations. But if you are the first to uncover an association, rigorously ask yourself the questions that are listed in Table 9.1 and report the results cautiously.

For example, Habu and colleagues conducted a study to assess whether administration of vitamin K2 would prevent bone loss in women with viral cirrhosis of the liver. After the study was completed, 40 of the 43 original participants agreed to participate in a longer trial. The investigators found that women who had been randomized to vitamin K2 in the original study were significantly less likely to develop hepatocellular carcinoma (OR 0.13; 95% confidence

Tip

Even after statistical adjustment, you can still have residual confounding.

In assessing whether your association could be due to confounding, consider both unknown (or unmeasured) and known confounders. Remember that only randomization can eliminate confounding due to unknown confounders (Section 2.4).

Even for known confounders, statistical adjustment using multivariable analysis is an imperfect method of eliminating confounding. This point is often missed by investigators who mistakenly believe that because they have included a potential confounder in their multivariable model, they have eliminated confounding due to that variable. But statistical adjustment for known confounders is never perfect. Your measure of the confounder may not perfectly capture the underlying confounder. Your model may not perfectly fit your data (no model does!). Therefore, even though you have “adjusted” for a confounder by including it in your model, you may still have residual confounding.