Skip navigation

Acupuncture for idiopathic headache

 

Clinical bottom line

There is no evidence that it works, and not a single trial was of sufficient quality or validity and size, and was positive. We need to beware of incorrect conclusions from systematic reviews.


A Cochrane review [1] of acupuncture for idiopathic headache concluded:

“overall, the existing evidence supports the value of acupuncture for the treatment of idiopathic headaches. However, the quality and amount of evidence are not fully convincing.”

Methods of the review

A number of relevant databases were searched to identify randomised, or quasi-randomised trials of acupuncture for the treatment of idiopathic headache. Valiant efforts were made to grade trials for quality, validity and appropriateness of the application and method of acupuncture used, but these were not taken into consideration when evaluating the effectiveness of acupuncture.

Results

The 26 included trials were diverse and some had serious methodological flaws. Several different control interventions were used in the trials. Meta-analysis was not appropriate and was not conducted.

Markers of higher quality and validity used were: (i) a quality score of three or more out of five (9/26 trials), (ii) an internal validity score of four or more out of six (7/26 trials), (iii) or a score of 70% or more for the appropriateness of the method and application of acupuncture (10/26 trials). Studies meeting the individual criteria for high quality, or validity or appropriateness of acupuncture were described. It may have been more helpful to identify trials which scored highly for all three criteria, and to put more weight on their results in the discussion and conclusions of the review. Only five of the 26 included trials scored three or more for quality and scored four or more for validity [Ho 1999; Hansen 1985; Henry 1986; Tavola 1992; White 1996]. They all compared acupuncture with sham acupuncture. Three assessed patients with tension-type headache [Hansen 1985; Tavola 1992; White 1996] and one assessed migraine [Ho 1999].

Acupuncture versus sham acupuncture

Seventeen of the 26 included trials used sham acupuncture as control. These are described in Table 1. Quality and validity scores were generally low. Three double blind studies were considered to have no interpretable data, either because of baseline differences between treatment groups or high rates of loss to follow up.

Table 1: Trials using sham acupuncture


Trial author

Type of headache Number of patients Blinding Quality score (max 5) Validity score
(max 6)
Comments
Baust Migraine 44 DB 3 3 Insufficient data presented
Hansen Tension 25 DB 3 4 Short duration, poor outcome; limited clinical relevance. Appropriateness of acupuncture 70%; no information on use of medication, intensity, duration or frequency of attacks.
Henry Migraine 30 DB 3 4 Appropriateness of acupuncture 45%; Ad Hoc definition; did mention information on improved intensity, duration or frequency of attacks. Follow-up data uninterpretable
Ho Migraine 52 DB 3 4 Appropriateness of acupuncture 85%; n=52; IHS criteria; no information on use of medication, intensity, duration or frequency of attacks.
Johansson Tension 33 DB 2 2 No numeric data presented. Not interpretable
Kubiena Migraine 30 DB 3 2.5 Large number of early drop-outs. Questionable validity; negative trial. No interpretable data
Lavies Various 12 DB 4 2.5 Baseline group differences in favour of acupuncture; no between group statistics. No interpretable data
Pintov Migraine 22 DB 2 2 Alternate allocation (quasi-randomised); selection bias possible
Tavola Tension 30 DB 3 5 Rigorous. Appropriateness of acupuncture 80%; Ad Hoc criteria; no information on intensity or duration of attacks.
White Tension 10 DB 5 4.5 Baseline group differences in favour of acupuncture; small group size (n=10); pilot study. Questionable validity
The studies below are not double-blind
Ceccherelli Migraine 30 Pt 1 3.5 Only followed-up responders. Questionable validity: cannot be strong positive. Positive bias to follow-up data
Dowson Migraine 48 Pt 2 3 Major reporting deficiencies; no follow-up data; unusual analysis
Heydenreich (a) Migraine 40? No blinding 1 2 Questionable validity. Not blind, possible observer bias
Shi Various 34 Pt 1 2.5 Major reporting deficiencies; no diagnosis data; no between group statistics
Vincent Migraine 32 Pt 1 4 Single (patient) blind; individual patient data
Weinschutz 1993 Migraine 40? Pt 1 2 Rigorous but poor reporting. Numbers almost identical to Weinschutz 1994 for every outcome etc. Duplicate?
Weinschutz 1994 Migraine 41? Pt 1 2 Rigorous but poor reporting. Numbers almost identical to Weinschutz 1993. Duplicate?
DB: double blind
IHS: International Headache Society
Pt: single (patient) blind

Table 2 shows the results of trials when information was segregated by different potential biases within the studies. Results were influenced by the quality, validity, and extent or absence of blinding. Trials of lower quality produced more positive results, as did studies that were not double blind.

Table 2: Outcome according to source of bias. Trials of acupuncture versus sham acupuncture


Potential source of bias Positive statistically Negative
No source of bias considered 7 7
Randomised 6 7
Quasi-randomised 1 0
Double blind 3 4
Not double blind 4 3
No blinding 1 0
Patient blind trials 3 3
Observer blind 0 0
Reporting quality of 3 or more 1 4
Reporting quality of 2 or less 6 3
Validity score of 4 or more 2 3
Validity score of 3 or less 5 4
Reporting quality of 3 or more & IVS score of 4 or more* 1 3
Reporting quality of 2 or less & IVS score of 3 or less* 5 3
Number of patients 40 or more 2 4
Number of patients 39 or less 5 3
14 trials provided outcome data which could be interpreted
*1 study with a quality score of 1 and validity score of 4 was excluded from this analysis
Negative: studies lacking significant differences between treatments, or acupuncture to be less effective than control

The method of acupuncture used in [Henry 1986] was inappropriate (score 45%). This means there are two negative trials [Tavola 1992] [Ho 1999] and one positive [Hansen 1985] trial with adequate methodology for acupuncture compared with sham acupuncture in the treatment of tension-type headache or migraine. Despite scoring reasonably well for quality, validity and appropriate use of acupuncture, these trials still had problems.

No information was provided about the intensity or duration of headache attacks, and only one [Tavola 1992] mentioned the frequency of attacks. These are important outcomes to patients, since headaches can occur frequently with detrimental impact on quality of life. Trials omitting this information are of limited clinical relevance. The clinical relevance of the study by [Hansen 1985] was limited by its duration of treatment and follow-up, and its choice of poor outcomes (headache index reduction and derived responder rate). It also failed to list the inclusion criteria for patients [Hansen 1985]. Without this type of information it is difficult to assess the results of the trial.

Comment

The majority of trials included in this systematic review were small, often diagnostic criteria or inclusion criteria for patients entering a trial were poorly reported, no information was provided about the severity of headache before administration of study treatment, and in some patient characteristics differed greatly between treatment groups. Overall, trials were of poor methodological quality, low validity, and often the application of acupuncture was either inappropriate or could not be assessed. The general quality of reporting of information in the trials was poor, especially with regards to descriptions of drop-outs. There are no methodologically rigorous studies which assess clinically relevant outcomes.

Studies of higher quality and validity should be given more weight in systematic reviews because they are more likely to produce reliable results. This review is yet another example of how giving equal weight to poor quality studies can lead to erroneous conclusions. Recommendations cannot be made to implement the use of acupuncture in the treatment of idiopathic headache based on current evidence.

The bottom-line question is to ask how many trials were of acceptable quality, validity, and size? The answer is not many. How many of these were positive? None. The conclusion that the evidence supports acupuncture for idiopathic headache is zero, and the authors of this Cochrane review mislead us.

How should trials be conducted in acupuncture?

High quality, valid trials of adequate size are lacking in this field. There is no reason why studies cannot be adequately randomised, double blinding is possible, and there are preferred, International Headache Society recommends clinically relevant outcomes. Diagnostic criteria should be clearly described, with randomised patients experiencing sufficient pain (moderate or severe) before administration of the intervention. Trials should have a sham acupuncture control. Dosing, duration and follow-up should be adequate and clearly described, as should information on the collection, reporting and severity of adverse effects.

References

  1. Melchart D, Linde K, Fischer P, Berman B, White A, Vickers A, Allais G. Acupuncture for idiopathic headache (Cochrane Review). In: The Cochrane Library, Issue 3, 2002. Oxford: Update Software.