• en

UK Science & Technology report

It is often reported that there is no evidence homeopathy works or that the current evidence base shows homeopathy is no better than placebo. Neither statement is correct.

Such misconceptions stem largely from two documents – the UK House of Commons Science and Technology Committee ‘Evidence Check 2: Homeopathy’ report, and the more recent Information Paper on Homeopathy produced by the Australian National Health and Medical Research Council (the “Australian Report”).

The UK Science & Technology: Evidence Check 2 report (EC2) was published in 2010 by a committee of 14 Members of Parliament (MPs)1. This report concluded that homeopathy works no better than placebo and that there should be no more research carried out into homeopathy.

Science & Technology EC2 report – Key Facts

  1. The report is not a scientific document and therefore should not be considered part of the scientific literature of used as evidence by decision-makers
  2. It is not just homeopaths who say it is flawed – the report has been widely criticised by people outside the homeopathic profession
  3. The committee of MPs excluded all evidence on homeopathy other than 5 systematic reviews and then based their conclusions on only 1 of these studies
  4. The report does not represent the view of UK Government – the Department of Health dismissed the report

Reliability of the report

As this document continues to be widely referred to, its reliability needs to be considered objectively. Although described by some as a ‘comprehensive review’ of the evidence, the ‘Evidence Check 2 report’ is not a scientific document – it is a report compiled by a committee of 14 MPs. No systematic scientific method was applied, it was not carried out by expert academics in the field and the choice of evidence included showed a disturbing bias – both in terms of written submissions and the choice of witnesses permitted to give oral evidence.

The multiple flaws in the EC2 report were significant enough to draw widespread criticism from fellow politicians who are familiar with how such Evidence Checks should be conducted:

  • 4 MPs voted on the report: 3 voted to ratify the report and 1 MP (Ian Stewart MP) abstained, dissenting from the report because he was concerned by the “balance of witnesses”
  • 70 MPs expressed their concern by signing an Early Day Motion (EDM 908)2
  • An independent critique by Earl Baldwin of Bewdley concluded that the report was “an unreliable source of evidence about homeopathy”.3 Earl Baldwin’s opinion is of particular interest as he served on the House of Lords Science and Technology Sub-Committee that inquired into complementary and alternative medicine in 1999-2000 and so was familiar both with correct S & T Committee procedures and the topic in question.

These and other problems have been reported by a dedicated website: http://www.homeopathyevidencecheck.org/.

What evidence does the report cover?

Reliability aside, a second pertinent issue is that the EC2 only considered clinical evidence. Even then, the only clinical evidence considered was on the efficacy of homeopathy not effectiveness i.e. they only looked at trials testing whether homeopathy works under tightly controlled, artificial experimental conditions, not studies testing whether it works on ‘real patients’ under real world clinical conditions.

Only five systematic reviews of randomised controlled trials (RCTs) were considered by the committee4,5,6,7,8 . From this evidence the four meta-analyses which reached broadly positive findings in favour of homeopathy were excluded,4,5,6,7 based entirely on the testimony of Prof Edzard Ernst9 who stated that, in his opinion, three were out of date and one should actually be considered negative. The only study that Ernst did not criticise was ‘The Lancet study’ by Shang et al. published in 20058, which he described as reaching a “devastatingly negative overall conclusion”.

Reliability of the Lancet study

As the EC2 report’s conclusion was effectively based solely on the Shang et al. study, once again the quality and reliability of this evidence becomes of paramount importance.

Multiple concerns have been raised about the Shang et al. study, particularly the fact that its conclusions were based on only 8 trials out of the 110 available to the authors at the time and that it fails a sensitivity analysis10 i.e. if you change just one of the 8 trials they chose to include in their analysis, the result is reversed, showing that homeopathy works beyond placebo. This completely undermines the reliability of the paper’s reported findings.

Furthermore not one of those 8 trials used involves individualised homeopathic treatment – the form of homeopathy considered to be ‘usual care’.

 The EC2 report is now woefully out of date

Although the EC2 was published in 2010, the report based its conclusions on systematic reviews published up to 2005. Prof Ernst also states in his submission that his arguments (against homeopathy) were based on evidence published up to 200511. That means that the evidence discussed in 2010 was already at least five years old.

As we take a fresh look at the evidence base for homeopathy in 2017, it is clear that the field of homeopathy research has progressed significantly since the Evidence Check 2, including publication of more recent systematic reviews.

For example, a review by Mathie et al.12 published in 2014 found that homeopathic medicines, when prescribed during individualised treatment, are 1.5 to 2.0 times more likely to have a beneficial effect than placebo. This study includes 151 placebo-controlled randomised trials – 41 more than Shang’s team identified, but which would have met their inclusion criteria if they had been available at the time.

This demonstrates the extent to which the now-12 year-old Shang et al. paper, which covers only 73% of eligible trials, has been superseded by the Mathie et al. 2014 paper – evidence of the highest academic quality which did not exist at the time of the Evidence Check 2.

An even more recent systematic review by Mathie et al.13 published in 2017 included 75 double blind, randomised placebo controlled clinical trials of non-individualised homeopathic treatment for a broad range of conditions. Mathie et al. found a small beneficial effect for non-individualised homeopathic treatment that was statistically significantly different from placebo. However, this pooled overall result only partially withstood rigorous sensitivity analysis, where meta-analyses were performed on 18 different sub-groups of trials including sample size, potency, acute/chronic conditions etc. The overall quality of the evidence prevented decisive conclusions, while the results’ positive trend indicates the need for more and higher-quality trials.

UK Government position

The government response to the Science & Technology Committee’s report was published by the Department of Health in July 201014.

The government refused to ban homeopathic products based on and identified homeopathy as a recognised and widely used system of medicine across the EU. The response emphasised patient choice as a key reason for continuing to fund homeopathy on the NHS.

ReferencesLess

  1. Evidence Check 2: Homeopathy, Report by the House of Commons Science and Technology Committee, February 2010 | Link
  2. UK Parliament Early Day Motion 908 | Link
  3. Earl Baldwin of Bewdley, June 2010: Observations on the report Evidence Check 2: Homeopathy by the House of  Commons Science and Technology Committee, February 2010 | Link
  4. Kleijnen, J., Knipschild, P. & ter Riet, G. Trials of homeopathy. BMJ, 1991; 302:960 | PubMed
  5. Linde, K. et al. Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet, 1997; 350: 834–843 | PubMed
  6. Linde, K. et al. Impact of study quality on outcome in placebo-controlled trials of homeopathy. J. Clin. Epidemiol., 1999; 52, 631–636 | Pubmed
  7. Cucherat, M., Haugh, M. C., Gooch, M. & Boissel, J. P. Evidence of clinical efficacy of homeopathy. A meta-analysis of clinical trials. HMRAG. Homeopathic Medicines Research Advisory Group. Eur. J. Clin. Pharmacol., 2000;56: 27–33 | PubMed
  8. Shang A, Huwiler-Muntener K, Nartey L, et al. Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet, 2005; 366: 726–32 | PubMed
  9. House of Commons Science and Technology Committee – Evidence Check 2: Homeopathy, Fourth Report of Session 2009-10. (February, 2010) Page 18, Point 65 to 69 (p.22 of PDF) | Link
  10. Lüdtke, R. & Rutten, A. L. B. The conclusions on the effectiveness of homeopathy highly depend on the set of analyzed trials. J. Clin. Epidemiol., 2008; 61: 1197–1204 | PubMed
  11. House of Commons Science and Technology Committee – Evidence Check 2: Homeopathy, Fourth Report of Session 2009-10. (February, 2010) Memorandum submitted by Professor Edzard Ernst (HO 16), Evidence Ev 26 (p.85 of PDF) | Link
  12. Mathie, R. T. et al. Randomised controlled trials of homeopathy in humans: characterising the research journal literature for systematic review. Homeopat. J. Fac. Homeopat., 2013; 102: 3–24 | PubMed
  13. Mathie, R. T. et al. Randomised, double-blind, placebo-controlled trials of non-individualised homeopathic treatment: systematic review and meta-analysis. Systematic Reviews, 2017; 6:63 | PubMed
  14. The UK government response to the Science & Technology Committee’s report | Link