Reliability versus validity

  Diagnosis in psychiatry has moved towards a criterion-based system (see the diagnostic criteria for depression in Chapter 1). The traditional approach of pattern recognition and reflective empathy informed by extensive familiarity with normal and abnormal behaviours has been replaced by a process of carefully listing features of the disorder that are present. The change was a response to unacceptable variations in diagnostic practice. The new diagnostic system (laid out in the Diagnostic and Statistical Manual – DSM III, now DSM IV) also strove to avoid relying on the psychiatric theories which had caused such conflicts in the past. Whether one really can have an entirely ‘atheoretical’ diagnostic system is, of course, open to debate.

  The new system emphasizes reliability (i.e. ensuring that different psychiatrists faced with the same symptoms will always come to the same diagnosis) more than it does validity (i.e. ensuring that patients with a particular diagnosis will have similar outcomes and responses to treatment). The goal would be, of course, maximal reliability and maximal validity. Good reliability does not, however, necessarily guarantee good validity. The fact that we all agree on the defining characteristics doesn’t mean it really is ‘something’. For example, 17th-century witch-finders were very reliable – they all agreed on the tell-tale signs and so consistently agreed on who was a witch before they burnt her. We would not now say that they had really ‘identified’ a witch, because we don’t believe in them, but their methods were certainly very reliable.

  Reliability can mistakenly imply validity so that a condition gets accorded the status of a diagnosis essentially because psychiatrists can agree on how to define and recognize it. I have already mentioned a couple of these controversial diagnoses – social phobia and ADHD – but there are several more which really stretch credibility. Nicotine and caffeine ‘use disorders’ are now both official psychiatric disorders, but few of us would consider these mental illnesses. Similarly there is a range of behavioural patterns which have acquired the highly questionable status of a diagnosis (and therefore may receive ‘treatment’). An example is adolescent ‘oppositional defiant disorder’, which is suspiciously close to the description of a difficult teenager who simply refuses to do what his parents want.

Psychiatric gullibility

  Psychiatrists on the whole are trusting souls. They tend to take their patients’ stories at face value. This was vividly demonstrated by the psychologist David L. Rosenham’s famous study, ‘Being Sane in Insane Places’. In 1973 he got eight volunteers to go to emergency rooms in America complaining of a voice in their head which said ‘empty’, ‘hollow’, or ‘thud’. All eight were admitted to psychiatric units where they then behaved absolutely normally. The most amazing thing was that they stayed in hospital for an average of just under three weeks each before they were discharged. Even worse, most of them got a diagnosis on discharge of ‘schizophrenia in remission’. Not surprising then that there is such a call for reliability.

  So there are several forces acting on psychiatry (including the natural curiosity of researchers) which threaten continued expansion. Whether this is a desirable development is one that should not be left to the profession alone to decide but requires debate within the broader society (i.e. you).