Skip navigation

Using the Result

Bandolier 27 carried a report on an article about spectrum bias in diagnostic tests (the propensity of tests to change in sensitivity or specificity depending upon populations studied). It also suggested that, since sensitivity and specificity are less than user-friendly, what we needed was a simpler way of doing things - so we suggested that a number-needed-to-diagnose (NND) might be a useful invention.

Did we start a debate? We have had many letters and calls this week. Some have argued that sensitivity and specificity are absolutes, while others have suggested that warning about spectrum bias is timely. Some like NND, others think it a load of rubbish (and Bandolier didn't help by getting the definition of sensitivity wrong in the article: we agree with the rest of the world that sensitivity is defined as true positive divided by true positive plus false negative).

Perhaps it is indicative of the difficulties that exist in this area that this has easily been the single most contentious issue ever run in Bandolier .

A number of correspondents have asked why we did not mention the use of likelihood ratios for diagnostic tests - suggested for some time as a more effective way of using diagnostic tests. Since this is mea culpa time, the answer is simple - Bandolier forgot! Interesting, though, was how few of an audience of producers of diagnostic test results, and of users, knew about it. So, with its heart in its mouth, Bandolier has another go.

Data

Let's use the data on smoking and smoking tests mentioned in Bandolier 27 [1]. The researchers examined a general practice data base in Belfast, and examined 591 patients with a history of coronary heart disease; 153 were self-reported smokers. They were part of a randomised trial of smoking cessation.

Four biochemical tests of smoking status were used, the best of which were urinary measurements of nicotine metabolite (with or without creatinine correction). They also identified a small group of `smoking deceivers' - people who said they did not smoke, but who had urine nicotine metabolite concentrations typical of those of smokers.

Sensitivity and specificity are given in the table for all results and for results excluding smoking deceivers.
Urinary nicotine metabolite (200 g/L) Urinary nicotine metabolite/creatinine (20 g/mmol)
All results:
Sensitivity (%) 98.4 100.0
Specificity (%) 93.6 92.7
Positive LR 15 14
Negative LR 0.017 <0.001
Excluding deceivers:
Sensitivity (%) 98.4 100.0
Specificity (%) 99.0 98.1
Positive LR 98 53
Negative LR 0.016 <0.001

Likelihood ratio

The likelihood ratio (LR) can be calculated from the sensitivity and specificity of a test expressed as ratios rather than percentages. It expresses the odds that a given finding would occur in a patient with, as opposed to without, the target disorder or condition. It is derived as:

LRpos = sensitivity / (1 - specificity)

With the LR above 1, the probability of the disease or condition being present goes up; when it is below 1 the probability of it being present goes down, and when it is exactly 1 the probability is unchanged.

LR can also be calculated for the negative, as well as the positive. To find the odds that a given finding would not occur in a patient without, as opposed to with, the target disorder or condition, LR is derived as:

LRneg = (1 - sensitivity) / specificity

Using the LR for tests of smoking status

Positive and negative LRs are calculated in the table. They are used by reference to the nomogram. The pre-test probability is a simple calculation - it is what proportion of this population is known to smoke before any tests are done; it is 153/591, or about 25%.
Using that as the starting point, the LRs can be used to give the post-test probability that any patient is a smoker when the result of the test is known. Three lines have been put on the nomogram for three possible circumstances:

  1. if the test is positive, using all results for either test gives a post-test probability of a patient being a smoker of about 85%.
  2. if the test is positive and we are happy to use data with smoking deceivers excluded (which seems reasonable), then a positive urinary nicotine metabolite test has an LR of 98, and we can be about 97% sure the patient is smoker.
  3. for any negative test, LRs are very low and we can be more than 99% sure that the patient does not smoke.

Some comments

Likelihood ratios are useful in several ways. Firstly they seem comprehensible and can be used to support or exclude a diagnosis. They can also be used sequentially so that the post-test probability after one diagnostic test can be used as a pre-test probability for the next. LRs can also be calculated at several levels of a test which produces numerical data.

The other thing that the nomogram indicates is just how good tests need to be to be useful in screening. Screening is usually done when the pre-test probability is low - often less than 1%. Here even a good test gives a low post-test probability. Just for fun, use the nomogram with a few examples you make up.

Of course, one needs some information on the sensitivity and specificity of tests, or LRs, and pre-test probabilities. If readers think this approach is useful, then Bandolier will try and find some examples.

Postscript

Just in case anyone was wondering what happened in the trial of smoking cessation in Belfast, there was no evidence that a health visitor encounter every 4 months for 2 years made much difference.

Useful references:

  1. GPR Arnold, M Cupples, A McKnight, T Linton. Measurement of markers of tobacco smoking in patients with coronary heart disease. Annals of Clinical Biochemistry 1995 32: 201-7.
  2. DL Sackett. Evaluation of clinical method. Oxford Textbook of Medicine. OUP, 3rd edition, 1996; 15-21.
  3. DL Sackett, RB Haynes, GH Guyatt, P Tugwell. Clinical Epidemiology. Little, Brown and Company, 2nd edition, 1991.
  4. R Jaeschke, GH Guyatt, DL Sackett. Users' Guides to the Medical Literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid? Journal of the American Medical Association 1994 271: 389-91.
  5. plus : Journal of the American Medical Association 1994 271: 703-8.



previous story in this issue