3 Evidence in practice

Introduction

A common concern about the emergence of evidence-based medicine (EBM) is the suggestion that medicine will be reduced to an automated “recipe book” method and the potential for skill and excellence in clinical practice will be lost. It is true that modern demands of clinical governance require the achievement of uniformly high standards of care. One of the means of implementing and maintaining standards of governance is through the development of clinical guidelines. Clearly, these need to be based on good quality evidence.

Evidence-based practice is in fact the converse of restrictive recipe book care. The culture of EBM has emerged with a shift in the paradigm of medical teaching from a factual basis, learnt by rote, to one of enquiry and thinking in terms of probabilities; moving from certainties to measures of uncertainty. Thus, the clinician challenges established dogma and strives for new answers to old problems. This approach protects the clinician from professional boredom and means that a blind eye is not turned on observations that do not fit the factual paradigm. It allows for progress.

None of these prescriptions limit the freedom of the clinician to strive for individual excellence in the diagnosis, prognosis and offer of therapies to their patients who understand the trade-off of risk and benefit for any given treatment. Archie Cochrane’s view was simple: if the taxpayer was to fund public health services, then finance should be provided only for interventions whose effectiveness was beyond doubt.1

Evidence may be applied in three main domains: individual patient care, policy making and research prioritisation.

Individual patient care

Modern clinical practice requires the setting of clinical standards for governance and this is achieved through guideline development. Guidelines are used for setting local standards for the delivery of care. These standards can then be the subject of regular audit to ensure the guidelines are being followed.

Guideline development should be an evidence-based process and organisations such as the Scottish Intercollegiate

Guideline Network (SIGN)2 have refined the process with graded levels of evidence for each guideline. Guideline development also feeds back to research prioritisation when evidence is absent. Systematic reviews are valuable in providing accessible unbiased summaries of evidence for those developing clinical guidelines.

Evidence may sometimes contradict current practice. The use of antibiotic ointment and an eye pad remains standard management for corneal abrasion. A few trials have examined the effectiveness of this measure and have failed to show any benefit (see Chapter 21). So why is this evidence not applied? There is much debate on what factors actually influence and change practice. A key issue remains the need for clinicians to be aware of the evidence. The importance of dissemination of research findings is critical. The Cochrane Collaboration, the numerous agencies for setting standards of care and publications such as Clinical Evidence are all dedicated to tackling this issue.

Bedside EBM, as advocated by David Sackett,3 means asking questions relevant to the individual patient during a ward round and using live online searches to seek answers to those questions. This approach is probably not compatible with the accelerated rounds used by surgeons but it is an important training methodology that teaches training clinicians to question the evidence base for every clinical situation.

Patients or consumers of health care are increasingly aware of the importance of evidence in guiding their choice of treatment. Consumer networks are becoming increasingly vocal and demand access to evidence in lay terminology that allows them to make informed choices.

Policy making

Major decisions are made by institutions such as academies, colleges and non-governmental organisations, as well as ministries of health, which require reliable information or good evidence as their basis. Such decisions often follow rather than precede changes in practice. Thus, standards are set based on what is observed to be common practice and therefore acceptable by consensus. How then do changes occur? Often, a new treatment option emerges with much marketing support from the manufacturers or having been pioneered by individuals who have influence

and reputation in the profession. Acceptance of the new intervention as an important advance occurs and those who fail to adopt the change are in danger of being regarded as obsolete and outdated. Sometimes trials are then undertaken to justify the change in practice that has already occurred.

This probably describes the process of change from intracapsular to extracapsular cataract extraction in the 1980s. The shift was an expensive and challenging adventure for ophthalmologists in the more advanced economies but presented insurmountable obstacles to those in poorer parts of the world. The evidence indicates that there is not much difference in risk of adverse outcome between the two techniques compared to the impact of any cataract surgery versus none (see Chapter 32).4 Forcing experts in the old technique to change may actually have done harm by reducing the throughput of operations and causing poorer outcomes as a result of a long learning curve for the new technique. Ironically, it is not uncommon for the extracapsular procedure to be performed without the implantation of an intraocular lens where patients cannot afford to purchase the lens. When funds are insufficient, an operation that was previously superseded by the intracapsular technique (an improvement because of the absence of posterior capsular opacification (PCO)), is almost inevitable in an extracapsular procedure without an intraocular lens. It was the intraocular lens, whether in the anterior or posterior chamber, that had the critical influence on outcome. Good quality trial evidence on the longer term outcomes of cataract surgery and PCO rates in the developing world setting where there are no Nd:YAG lasers is still lacking.

There are worse examples where irrevocable harm has been done. Diethylcarbamazine (DEC) was widely distributed in regions where onchocerciasis was endemic on the basis that it was known to be a powerful microfilaricide. Death of microfilaria in the eye caused uveitis (the Mazzotti reaction) but it took years to realise that DEC actually precipitated blindness by causing an inflammatory optic neuropathy.

These examples illustrate the importance of the use of evidence in informing policy. This includes organisation of care as well as specific interventions. It embraces decisions on public health policy, sanitation and health promotion. When evidence is found to be absent, new studies are needed before change in practice is implemented.

Research prioritisation

Within the NHS, there is a debate on the value of evidence and the extent to which research is useful in guiding practice. Although there are some who feel that scarce healthcare resources should not be wasted on research but should be committed only to patient care, most

recognise the value of evidence that guides practice. But research resources are seriously limited and providers of funding need guidance on the evidence that is most needed.

Critically important in making funding decisions is consideration of what is already known and what is not known about the effectiveness of interventions. The description of the evidence base needs a scientific and unbiased method, which is provided by systematic reviews.

It is the policy of the UK’s Medical Research Council5 to require applicants to include the results of a systematic review of the effectiveness of an intervention in their application for funding for a new trial. Not only does the review provide the detailed summary of the existing knowledge but it educates the applicant about the pitfalls and difficulties encountered by previous research. The exact knowledge deficit in terms of the reference population, the types of intervention and the most relevant and reliable way of measuring outcome can be determined.

The WHO has adopted the SAFE strategy as part of the VISION 2020 initiative to eliminate blinding trachoma by the year 2020.6 This is an example of a policy preceding the systematic description of the evidence base. However, this pragmatic policy has helped focus resources. Systematic reviews of each of the components will identify the key questions in implementing this strategy. None of the interventions yet have sound evidence on which any acceptably precise estimate of effectiveness and cost effectiveness can be made (see Chapter 20). The WHO and other international non-governmental organisations can use systematic reviews of these questions to make decisions about priorities in funding new research to fill the existing gaps in the evidence base for their policies.

Quality of evidence

Randomised controlled trials (RCTs) and systematic reviews of RCTs are considered to provide the highest level of evidence. This is because the process of randomisation, innate in the design of a properly conducted RCT, is the best way of controlling for both known and unknown confounding factors that may determine differences between control and intervention arms of a study. Thus, observed differences in outcome can be attributed to the intervention alone.

Observational studies can deal only with known confounders in the design and/or analysis. When no such confounding is present, observational studies and RCTs may produce the same results, as has been observed by Benson and Hartz.7 Because so many factors may be present that consciously or unconsciously influence clinicians in the conduct of studies of effectiveness of new interventions, clinician scientists need to adopt the scepticism of

laboratory scientists in attributing cause and effect before believing and implementing the findings of such studies.

It is for this reason that the Cochrane Collaboration, the BMJ’s Clinical Evidence and the authors of this book have decided to deal only with RCTs. It leaves out a huge body of lesser evidence that may or may not influence our practice but there must be doubt of the value of summarising studies of effectiveness other than those that use the optimum design.

A systematic method is needed for judging the quality of trials selected for summary. This is an evolving art but current policies advise considering allocation concealment, masking of participants and observers of outcome, completeness of follow-up in the randomisation groups and intention to treat analysis.8

Many now agree that there is no satisfactory method for giving trials a single trial quality score. A trial may be excellent in all but one critical domain and hence score well despite being fundamentally flawed. Thus each domain should be considered independently. More can be read about these issues on the Cochrane website at http:// www.cochrane.org.

Limitations of trial evidence

There are two important areas where RCTs fail to inform clinicians. The first concerns the external validity of trials. Strict inclusion and exclusion criteria employed in many studies mean that the effectiveness of interventions on population subgroups excluded from studies remains unknown. Inevitably, vulnerable subgroups such as pregnant women and children often remain in an evidence vacuum.

In addition, it is recognised that participants in RCTs fare better, even in the control or placebo arm, than in the real world. Thus an important addition to the armoury of evidence is the outcome study, where a large representative and unselected or sequential series of interventions are monitored for their outcome. Good examples of this are the national surveys of cataract surgery outcome conducted by the Royal College of Ophthalmologists.9 These studies are important in establishing standards for care that reflect reality more closely than RCTs.

The second concern is that trials cannot inform the clinician of rare but serious adverse events. Trials are rarely large enough to answer these important questions and followup is seldom long enough to detect late but serious complications. For drugs there are mechanisms for reporting such events to a central authority but no such mechanisms are uniformly in place for surgical techniques. An example of this is the concern that has arisen about the long-term safety of mitomycin in the control of wound healing in glaucoma

surgery. Surveillance systems are developing, such as the BOSU for monitoring rare but important adverse outcomes.10

There is no doubt that better and more reliable information systems are required to deal with this issue but one might expect with the development of reliable methods of information retrieval, systematic audit of outcomes will help.

Conclusion

The importance of the emergence of evidence-based medicine is freeing the evolution of clinical practice from the dangerous biases of the medical market and fashion to a protocol-driven scientific process. Contrary to the fears of limiting clinical freedom, evidence-based practice releases clinicians from the factual paradigm to understanding the importance of probability and levels of uncertainty. This opens the mind of the clinician to question continually the basis of their practice and to the possibility of progress.

Enormous challenges remain and there are many areas of our practice that present significant difficulties to clinical researchers who wish to fill the gaps in the evidence revealed in the following chapters; not least, the huge difficulty in finding funding for clinical research.

At least two of the annual meetings of the international Cochrane Collaboration have been addressed by the host country’s minister of health. On both such occasions, the ministers have agreed that the importance and value of evidence to inform health policy is essential but that it rarely happens. We hope that this wisdom will eventually prevail on those who provide funding for clinical research.

References

1.Cochrane AL. Effectiveness and Efficiency. Random Reflections on Health Services. London: Nuffield Provincial Hospitals Trust, 1972.

2.Scottish Intercollegiate Guideline Network. http://www.sign.ac.uk

3.Sackett DL, Straus SE. Finding and applying evidence during clinical rounds: the “evidence cart”. JAMA 1998;280:1336–8.

4.Snellingen T, Evans JR, Ravilla T, Foster A. Surgical interventions for age-related cataract. In: Cochrane Collaboration: Cochrane Library. Issue 2. Oxford: Update Software, 2002.

5.MRC guidelines for good clinical practice in clinical trials: 1998. London: Medical Research Council, 1998, Appendix 2:2.4.

6.Cook JA. Trachoma and the safe strategy. J Community Eye Health http://www.jceh.co.uk/journal/32_1.asp

7.Benson K, Hartz AJ. A comparison of observational studies and randomized controlled trials. N Engl J Med 2000;342:1878–86.

8.Juni P, Altman DG, Egger M. Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ 2001;323: 42–6.

9.Desai P, Minassian DC, Reidy A. National cataract surgery survey 1997–8: a report of the results of the clinical outcomes. Br J Ophthalmol 1999;83:1336–40.

10.Foot B, Stanford M, Rahi J, Thompson J. The British Ophthalmological Surveillance Unit: an evaluation of the first 3 years. Eye 2003; 17:9–15.

Why does this section focus on primary care?

There are two main reasons for focusing on primary care. Firstly, primary care clinicians are, in many parts of the world, the first port of call for those suffering from ocular symptoms. Eye-related consultations represent a significant proportion of the general practitioners workload, representing an estimated 1 in 20 of all general practitioner consultations in economically developed countries such as Britain. Only a small minority of these consultations are for conditions that may require subsequent referral for specialist assessment and treatment and these conditions are covered in other sections of this book. The majority of patients can, however, safely be diagnosed and treated within a primary care setting and it is the management of these commonly encountered external ocular conditions

that forms the focus of this section. Secondly, as health systems in many parts of the world increasingly begin to focus on ways of preventing visual impairment, there is an emerging body of evidence, drawn from primary care, evaluating the role of screening interventions for ocular problems.

The main aim of this section is to describe and critically appraise the available evidence from trials from the Cochrane Eyes and Vision Group Register and published systematic reviews of randomised controlled trials of treatments for bacterial, viral and allergic conjunctivitis and screening interventions for visual impairment in older people. Contributors have also highlighted areas in which evidence is lacking and identified key questions that have yet to be addressed in relation to the management of these primary care problems.