Skip navigation

Meta-Meta-Analysis

Is cholesterol lowering beneficial?
Garbage in - garbage out?
Elephants and pygmies

To add to the 'King Kong versus Godzilla' arguments about whether a single large randomised trial is better than a meta-analysis of smaller trials we now have to add problems of discrepancies of meta-analyses themselves. The study of the value of cholesterol lowering and the effects on coronary heart disease [1] is the first Bandolier has found which looks at the results of different meta-analyses and tries to prise out nuggets of philosophical gold.

Is cholesterol lowering beneficial?


Twenty-three separate meta-analyses were found. Outcomes examined were those of total mortality, cardiovascular mortality and nonfatal cardiovascular disease. As the Table shows, the results of the analyses overwhelmingly supported cholesterol lowering for reduction in nonfatal cardiovascular disease and cardiovascular mortality, but not total mortality.

Event Number of meta-analyses Favours treatment No difference Favours control
Nonfatal cardiovascular disease 12 12 0 0
Cardiovascular mortality 15 10 5 0
Total mortality 18 3 14 1

Garbage in - garbage out?


Of the 23 analyses, eight were not supportive of benefit. Of those eight, three included non-randomised studies, and one didn't state whether it did or not. None of the 15 studies supportive of benefit included non-randomised studies. So here is one lesson learned again: for treatments, including non-randomised studies has to be justified.

The supportive meta-analyses were also generally better designed. Not only did they include only randomised studies, but had more explicit exclusion criteria that allowed a more direct evaluation of the effects of cholesterol lowering without confounding factors. None of the non-supportive meta-analyses included authors with meta-analytic expertise. An interesting post-hoc observation was that only 4 of 10 analyses in British journals were supportive, compared with 11 of 13 in non-British journals.

Elephants and pygmies


Faced with an elephant, half a dozen blindfolded pygmies will each come out with a different description of what it is. And when it comes to knowing the truth, blindfolded pygmies just about sums up where each of us is. The answers may be simple, but it is the questions that are difficult.

So if you want to know the truth about cholesterol lowering, the spread of questions my be as wide as whether cholesterol lowering reduces overall mortality to which cholesterol lowering regimen is best for my particular patient? The way in which information is gathered to provide the knowledge-base to try and answer each of these questions will be different.

So we should praise the fact that so many meta-analyses have been done, not bury meta-analysis because so many have been done. The problem, though, for the busy practitioner, is to have simple tools to hand to make sure that the many clinical decisions made every day are made as correctly as possible. For treatments, those tools will come best from systematic review and meta-analysis. How long before we see then first meta-meta-meta analysis?

Reference:


  1. DA Katerndahl, WR Lawler. Variability in meta-analytic results concerning the value of cholesterol reduction in coronary heart disease: a meta-meta-analysis. American Journal of Epidemiology 1999 149: 429-41.
previous or next story in this issue