Skip navigation
Link to Back issues listing | Back Issue Listing with content Index | Subject Index

Beware statistical outputs

From hypothetical to specific
Making decisions
Procedure-specific prescribing

Suppose we have an intervention in which the NNT is 3.3 compared with placebo. That tells us that for every 3.3 people given the intervention, one more would have a treatment-specific outcome than would have occurred with placebo. Put another way, 30% more patients would have the outcome with the intervention than with placebo.

Suppose we were told the relative benefit was 3.3. That tells us that with treatment the event occurs three times more often than it does with placebo.

In both cases, if we knew the average placebo response rate, we would know how many patients would have the outcome with the intervention. Without that knowledge, the relative benefit is particularly meaningless.

Look at Figure 1. It shows the relative benefit for a series of hypothetical studies in which the placebo response rate varies between 1% and 61%, and where 30% more patients have the outcome with intervention than with placebo. The relative benefit varies from 31 at 1% placebo response to 1.5 at 61%. In all these hypothetical trials the NNT is 3.3.


Figure 1: Relationship between relative risk and placebo response for 30% effect






From hypothetical to specific


A commentary criticising the use of NNTs in acute pain [1] tries to demonstrate that different procedures produce different results, based on comparisons of relative risk and seeking aid from the dubious ally of heterogeneity tests. The results on which it is based, for paracetamol 975/1000 mg in acute pain trials, are shown in Table 1.


Table 1: Results for paracetamol 975/1000 mg in acute pain studies after different procedures



Number of
Percent with at least half pain relief
Procedure
Trials
Patients
Paracetamol
Placebo
Paracetamol minus placebo
Relative benefit
(95%CI)
NNT
(95% CI)
Dental
9
1038
37
10
27
3.8 (2.8 to 5.1)
3.7 (3.1 to 4.5)
Oral
6
962
37
6
31
4.2 (2.8 to 6.3)
3.2 (2.8 to 3.7)
Episiotomy
4
597
69
38
31
1.8 (1.6 to 2.2)
3.2 (2.6 to 4.2)
Orthopaedic
4
214
54
29
25
1.9 (1.4 to 2.6)
4.1 (2.7 to 8.6)



Relative benefit in dental and oral surgery looks higher, at about 4, than the value of below 2 seen in episiotomy and orthopaedic procedures. NNT values were the same, with overlapping confidence intervals. The additional benefit of paracetamol 975/1000 mg in these four procedures, the percentage of patients with at least half pain relief with paracetamol minus that with placebo, is also the same (Table 1), as we expect from similar NNTs.

Making decisions


What differs is what happens when we do nothing. In dental or oral surgery only 10% of patients have at least 50% pain relief, while following episiotomy or minor orthopaedic procedures 30% do so. The effect of the analgesic is the same; the procedures are different.

Is paracetamol 975/1000 mg a sensible choice following episiotomy or minor orthopaedic procedures? It might be argued that it is, because most patients do well. It could also be argued that it is not a sensible choice in dental or oral procedures, because most patients do not do well.

Procedure-specific prescribing


Clearly, procedure-specific prescribing not only makes sense, but should be mandatory. Most therapy involves a package of interventions; what may be appropriate in one circumstance will not be in others.

The NNT just describes results from clinical trials. It is one way, but not the only way. Like almost all descriptors, it should not be used alone, but as a useful shorthand for busy people. In acute pain, efficacy of different analgesics in different procedures has been tested thoroughly [2]. Procedure does not affect efficacy in this particular case of clinical trials of analgesics in acute pain, and with usual provisos of sufficiency of data and limiting extrapolation.

Bandolier keeps on exhorting people to remember two important things: look at the data not just the statistics, and remember that clinical trials are not clinical practice. Using evidence does not mean putting your brain in neutral and accepting the evidence without considering appropriateness.

References:

  1. A Gray et al. Predicting postoperative analgesia outcomes: NNT league tables or procedure-specific evidence? British Journal of Anaesthesia 2005 94: 710-714.
  2. J Barden et al. Pain and analgesic response after third molar extraction and other postsurgical pain. Pain 2004 107: 86-90.

previous or next story