Skip navigation

How should you choose the intervention or who to do it?

Who patients are referred to, and how we make the choice, is a complicated question. If all the doctors were equally helpful to the patients, we would like to choose the intervention on the basis of which works best, and to choose where to have it done on the basis of least risk, shortest stay etc.

Managing outcomes

Gathering evidence to substantiate these choices is a vexed topic. American 'outcome managers' compare providers of the same service using banks of administrative data, such as computerised hospital discharge abstracts. How many patients who had a hernia operation at St Faith's died compared with St Elsewhere's? The league tables in the NHS are primitive by comparison but similar interpretations are made. The obvious flaw is that if St Faith's takes all the fit patients and St Elsewhere's takes all the unhealthy ones, then St Elsewhere's performance will appear worse. The case-mix of the hospitals is different, and this confounds simple comparison.

Similar confounding was seen using this approach to decide best treatment rather than best provider. The early comparisons of transurethral prostatectomy versus open resection selected open resection for the younger, fitter men (no randomisation). This biased selection led to a conclusion that open resection had lower postoperative mortality.

Non-randomised pitfalls

These pitfalls of using non-randomised data to help determine best treatment or best provider are dealt with in a paper from Toronto [1]. It looked at the impact of removing the appendix during open cholecystectomy, the impact on in-hospital fatality rate, complication rate and length of stay. To do this they used a central database which keeps the records of all the Ontario general hospitals. Of the (roughly) 200,000 cholecystectomies performed between 1981 and 1990, the surgeon took out the appendix as well in 7,846 patients.


The initial analysis showed a statistically significant reduction in mortality among the patients whose appendix was removed. This "paradoxical" result then evaporated with further analyses. These further analyses adjusted for confounding influences, comorbidity and non elective surgery. The bottom line was that in comparisons of low-risk groups, those under 70 having elective cholecystectomy, there was a statistically significant increase in nonfatal complications.


The important message is that the first analysis gave the wrong answer, that mortality was lower if the appendix was taken out. League tables which did not do further data-dredging would be misleading. The authors conclude "While no statistical adjustments can completely compensate for nonrandom case selection, routine restriction of any comparisons to low-risk subgroups also appears prudent to help determine whether persistent confounding is contributing to the apparent outcome differences between procedures or among providers".

If you want to choose the best treatment, or who does it best, in the ideal world you would do this on the basis of randomised comparisons. We are unlikely to have randomised comparisons for the "who does it best" comparison. Using audit data to help your decision may give the wrong answer. The best chance of getting the right answer would be to look at the results in the relatively fit and healthy.
  1. SW Wen , R Hernandez, DC Naylor. Pitfalls in nonrandomized outcomes studies. The case of incidental appendectomy with open cholecystectomy. Journal of the American Medical Association 1995 274: 1687-91.

previous or next story in this issue