Skip navigation

Testing a Test

The evaluation of a treatment is often difficult, but at least there is a clear outcome - either the patient gets better or does not.

When assessing a new diagnostic test, however, the situation may not be so simple. The successful "outcome" for a test may be either a positive or a negative result, depending upon whether a doctor wishes to confirm or exclude a diagnosis. Another possible way of assessing outcome is by whether the test affects the management of the patient: some test results, both positive and negative, will lead to a change in management, whereas in other circumstances a positive or negative result will lead the doctor to continue with the same management plan.

Diagnoses as drivers

The development of a new treatment is often seen as a very important driver of health service direction and costs. The development of new diagnostic tests is equally important. When a new diagnostic test is developed, for example the ability to identify an enzyme deficiency, then a new group of people will be identified as having a positive test result and this will initiate further diagnostic effort or treatment, even though there is not always good evidence that the treatment will be beneficial.

The introduction of commonly available tests to measure cholesterol has led to treatment costs of about £1.5 million per million population per year, even though our national policy is still not to screen the whole population for raised levels of cholesterol.

Sensitivity and specificity

Diagnostic tests can be classified as having either positive or negative results. Even those tests that have a continuous distribution can be defined as having an arbitrary cut-off point distinguishing positive from negative. This is done, for example, in Down's syndrome screening where a single cut-off point is taken as the level at which further investigation is justified.

People who have tests done either have or do not have an underlying disease or condition, and the relationship between these debatables can be most elegantly expressed by a two-by-two box.



Using data from real clinical studies in this simple two-by-two box, it is possible to assess the sensitivity and the specificity of the test.

The sensitivity of a test is the proportion of people with the disease who have a positive test result. The higher the sensitivity, the greater the detection rate and the lower the false negative rate.

The specificity of the test is the proportion of people without the disease who have a negative test. The higher the specificity, the lower will be the false positive rate and the lower the proportion of people who do not have the disease who will be unnecessarily worried or exposed to unnecessary treatment.

Predictive value

The positive predictive value of a test is the probability of a patient with a positive test actually having a disease. The negative predictive value is the probability of a patient with a negative test not having the disease.

While the sensitivity and specificity of a test are constant within the populations under test - and generally wherever the test is performed - the predictive value of a test result depends not only on the sensitivity of the test but also on the prevalence of the condition within the population being tested.

For instance, if a test has a sensitivity of 96%, then the positive predictive value varies with the prevalence as follows:-

The significance of a positive test may be very different in hospital than in general practice because the prevalence of disease is higher in the referred population. Hospital doctors frequently criticise general practitioners for "missing a diagnosis" because the significance of a positive test result is, for most tests, statistically different in hospital practice than in general practice.

The clinician's perspective

Clinicians judge tests primarily with respect to sensitivity and specificity, often underestimating the importance of predictive value when transferring the test from secondary to primary care.

However, the clinician does not necessarily seek a test with a high sensitivity and a high specificity.

When it is very important not to miss a diagnosis, a test with high sensitivity is chosen.

When it is very important not to create false positives, because of the serious consequences of a positive diagnosis, the clinician will give greater emphasis to assessing the test with respect to its specificity and will seek a test with higher specificity, even though there may have been a trade-off in the test with lower sensitivity than the alternative.

The purchaser's perspective

The purchaser's perspective obviously has to take into account the clinician's perspective, but the purchaser is likely to have different criteria for assessing a new test. The will include:-

Commissioning tests

Although district health authorities are having a growing influence on treatment and care services through their power to purchase and specify what services their population needs, they have relatively little influence over diagnostic services, even though those services have a major impact on their population and health care costs. GP fundholders have an influence on both imaging and laboratory services but the commissioning of diagnostic services is principally carried out by clinicians, a fact recognised in many Trusts by the move to set up an internal market with the budget for imaging and laboratory services being allocated to the clinicians who use those services. The choice of diagnostic services is therefore made by clinicians with the costs of diagnostic services being incorporated into the price of the treatment services which health authorities commission.

The approval of business and strategic plans of Trusts will offer the Regional Office an opportunity to identify and challenge a Trust business plan to make some major investment, for instance to buy a new CT scanner. However, as in so many areas of clinical practice, transformation comes about not by a small number of major investments but by a large number of small developments, and the development of new patient testing make this aspect of technology-creep one that will be even more difficult to control in future.

Evidence-based diagnosis

New diagnostic tests can be subject to rigorous evaluation and steps can be taken to appraise the evidence on which the decision to introduce a new diagnostic test is taken, but this approach is used all too rarely.

Diagnosis lies at the heart of clinical practice and developments in diagnostic services drive changes in health care. The direction and intensity of this drive is presently determined by those who invent, make or market diagnostic equipment, and by the clinicians who use new diagnostic opportunities. If best value for money is to be achieved, those who pay for health services must examine the evidence of every new diagnostic test much more rigorously.

J A Muir Gray MD FRCP (Glas) MRCGP FFCM

Director of Health Policy and Public Health

next story in this issue