Skip navigation

Sed Quis Custodiet Ipsos Custodes?

Real world motifs
Layered knowledge
Sticks and stones
Ultimate agreement

'Who is to guard the guards themselves?' is the translation of this quote from Juvenal's Satires. It was brought to mind by a thoughtful article by Neville Goodman [1] who asks 'who will challenge evidence-based medicine?' Goodman is concerned that EBM is in danger of becoming an unchallengable orthodoxy following its own political agenda.

Tough words, and possibly rightly tough. The underlying theme is that there is an ideological difference of opinion about EBM, and that there is probably no evidence that EBM provides better medical care in total than whatever we choose to call whatever went on before. After all, even EBMers themselves disagree all the time about the rights and wrongs of the technical bits of meta-analysis. EBM seems to be so statistical that you need a brain the size of a small planet even to begin to understand it. And finally, proponents ignore seemingly valid criticisms.

Real world motifs


Most of us live in a real world where these arcane arguments have little value. What we need are tools to help us get the job done best, fastest, and cheapest (probably in that order). We want a sort of holy trinity, involving evidence of effectiveness, value for money, and quality improvement.

Can evidence help us, or should we eschew everything called EBM because it may itself have problems? When EBM was defined [2] it was defined thus: 'evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients'.

Nothing wrong in that. Stated in a slightly different way, this is how Goodman's article also begins. It's not the use of evidence per se that irritates Goodman, but the special meaning that seems to derive from systematic reviews and meta-analyses, as if they have some magical power.

Layered knowledge


Meta-analyses may not always agree. This issue of Bandolier examines some of those disagreements in cholesterol lowering . But the upshot is that the meta-analyses generally do agree, and most of those not agreeing had included non-randomised trials - a no-no for therapeutics trials unless that is the only information you have .

So we have a meta-analysis of prophylactic antibiotics after basilar skull fracture. Only two of 12 included studies were randomised, and those two were small. So the bulk of the world's information comes from non-randomised studies. You have to make a decision whether it is the policy for your institution, or for this particular patient, to prescribe prophylactic antibiotics against possible meningitis. There is a balance of benefit and harm to be struck. Do you want this information (because that's all there is), or none?

Tricky, isn't it. Using only randomised studies makes sense when there are lots. It may not make sense when there are few or none, especially when they all say much the same thing. Having them in a meta-analysis is better than not knowing that the evidence is there, but doesn't make using the evidence any easier. It's the difference between a rule and a tool .

And if you want to try to make a difference in practice, you may have even less evidence than that. Look at the book review for the Promoting Action on Clinical Effectiveness (PACE) programme . No randomised trials at all - just good examples of how others have done it, plus experience. But that's evidence too, just a different type, with a different weight, to be used differently.

Sticks and stones


We look at crystal ball evidence relating to falls and hip fractures . Different sorts of evidence, with models of future health care needs - yet another sort of evidence. What we need to do is assess the evidence on risks, try and figure out who is most at risk of a fall or fracture, and try the various ways that have been shown (with various types of evidence) to be effective in preventing falls and fractures.

Ultimate agreement


This is not to dismiss Goodman's criticisms. Bandolier agrees with most of them. In the Oxford Dictionary 'cartel' is described as 'a combination of business firms to control production, marketing, etc. and avoid competing with one another'. There is a suspicion that there is an EBM cartel, which likes things its way, and where disagreement is suppressed. That would be bad if it were true. It is only by trying to do things differently that we learn that the rocks of yesteryear are the sands of today. More meta-analysis, and more disagreement, but with a constructive motif is what we need. And there's nothing wrong, and everything right, with a great big dollop of criticism, plus a regular changing (or checking) of the guards.

Reference:


  1. NW Goodman. Who will challenge evidence-based medicine? Journal of the Royal College of Physicians of London 1999 33: 249-51.
  2. DL Sackett et al. Evidence-based medicine - what it is and what it isn't. BMJ 1996 312: 71-2 and at http://cebm.jr2.ox.ac.uk/ebmisisnt.html


next story in this issue