UK Science & Technology report
It is often reported that there is no evidence homeopathy works or that the current evidence base shows homeopathy is no better than placebo. Neither statement is correct.
Such misconceptions stem largely from the House of Commons Science and Technology Committee ‘Evidence Check 2: Homeopathy’ report, produced by a committee of 14 Members of Parliament (MPs)1. This report concluded that homeopathy works no better than placebo and that there should be no more research carried out into homeopathy.
‘S & T’ report key points
- The report is not a scientific document and therefore should not be used as evidence by decision-makers
- It is not just homeopaths who say it is flawed – the report has been widely criticised by people outside the homeopathic profession
- The committee excluded all evidence other than 5 systematic reviews and based their conclusions on only 1 of these studies
- The report does not represent the view of UK Government – the Department of Health dismissed the report
Reliability of the report
As this document is widely quoted, its reliability needs to be considered. Although often referred to as a ‘comprehensive review’ of the evidence, the ‘Evidence Check 2 report’ is not a scientific document. No systematic scientific method was applied, it was not carried out by expert academics in the field and the choice of evidence included showed a disturbing bias – both in terms of written submissions and the choice of witnesses permitted to give oral evidence.
Such fundamental flaws have been widely acknowledged by those outside the homeopathic profession:
- 4 MPs voted on the report: 3 voted to ratify the report and 1 MP (Ian Stewart MP) abstained, dissenting from the report because he was concerned by the “balance of witnesses”
- 70 MPs expressed their concern by signing an Early Day Motion (EDM 908)2
- An independent critique by Earl Baldwin of Bewdley concluded that the report was “an unreliable source of evidence about homeopathy”.3 Earl Baldwin’s opinion is of particular interest as he served on the House of Lords Science and Technology Sub-Committee that inquired into complementary and alternative medicine in 1999-2000 and so was familiar both with correct S & T Committee procedures and the topic in question.
These and other problems have been reported by a dedicated website: http://www.homeopathyevidencecheck.org/.
What evidence does the report cover?
Reliability aside, a second pertinent issue is that the Evidence Check 2 report only considered efficacy not effectiveness i.e. they only looked at trials testing whether homeopathy works under tightly controlled, artificial experimental conditions, not studies testing whether it works on ‘real patients’ under real world clinical conditions.
They therefore excluded all observational studies (even high quality controlled studies) and only considered the five main comprehensive meta-analyses of randomised controlled trials (RCTs).4,5,6,7,8 From this evidence the four meta-analyses which found in favour of homeopathy were excluded,4,5,6,7 based on the testimony of Prof Edzard Ernst that, in his opinion, they were unreliable. This left only one study to inform the report’s conclusions – the comprehensive comparative meta-analysis known as The Lancet study by Shang et al. published in 2005.8
Reliability of the Lancet study
Considering that this single paper is the only major study ever to conclude that homeopathy is no more than a placebo effect, its role in the debate cannot be overstated, so one again quality and reliability are paramount.
Unfortunately multiple concerns have been raised about the Shang et al. study, particularly the fact that it’s conclusions were based on only 8 trials out of the 110 available at the time and that it fails a sensitivity analysis9 i.e. if you remove just one of the 8 trials they used in the analysis, the result is reversed, showing that homeopathy works beyond placebo. Furthermore not one of those 8 trials used involves individualised homeopathic treatment – the form of homeopathy considered to be ‘usual care’.
Reliability of the analysis is not the only problem with the Shang paper. As we take a fresh look at the evidence in 2014, we also need to consider how well this study reflects the entirety of today’s evidence base. A recent literature search by Mathie et al.10 has identified 151 placebo-controlled randomised trials which would have met the inclusion criteria for Shang’s review – 41 more than identified in 2005. This demonstrates the extent to which this 8 year old review, which now covers only 73% of the eligible trials, has become outdated.
A state of confusion
The chairman, Phil Wills, made the following statement during the committee hearings:
…….. there seems to be a little confusion about the nature of the work that we are doing, this is not an inquiry into whether homeopathy works or not. […….] I want to make that absolutely clear. I wonder if we can therefore start with you, Minister. Does the Government have any credible evidence that homeopathy works beyond the placebo effect? 11
The final report then concluded that homeopathy works no better than placebo.
UK Government position
The government response to the Science & Technology Committee’s report was published by the Department of Health in July 201012.
The government refused to ban homeopathic products and identified homeopathy as a recognised and widely used system of medicine across the EU. The response emphasised patient choice as a key reason for continuing to fund homeopathy on the NHS.
- Evidence Check 2: Homeopathy, Report by the House of Commons Science and Technology Committee, February 2010 | Link
- UK Parliament Early Day Motion 908 | Link
- Earl Baldwin of Bewdley, June 2010: Observations on the report Evidence Check 2: Homeopathy by the House of Commons Science and Technology Committee, February 2010 | Link
- Kleijnen, J., Knipschild, P. & ter Riet, G. Trials of homeopathy. BMJ, 1991; 302:960 | PubMed
- Linde, K. et al. Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet, 1997; 350: 834–843 | PubMed
- Linde, K. et al. Impact of study quality on outcome in placebo-controlled trials of homeopathy. J. Clin. Epidemiol., 1999; 52, 631–636 | Pubmed
- Cucherat, M., Haugh, M. C., Gooch, M. & Boissel, J. P. Evidence of clinical efficacy of homeopathy. A meta-analysis of clinical trials. HMRAG. Homeopathic Medicines Research Advisory Group. Eur. J. Clin. Pharmacol., 2000;56: 27–33 | PubMed
- Shang A, Huwiler-Muntener K, Nartey L, et al. Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet, 2005; 366: 726–32 | PubMed
- Lüdtke, R. & Rutten, A. L. B. The conclusions on the effectiveness of homeopathy highly depend on the set of analyzed trials. J. Clin. Epidemiol., 2008; 61: 1197–1204 | PubMed
- Mathie, R. T. et al. Randomised controlled trials of homeopathy in humans: characterising the research journal literature for systematic review. Homeopat. J. Fac. Homeopat., 2013; 102: 3–24 | PubMed
- Science & Technology Committee Evidence Check 2: Homeopathy, Session 2, Mon 30th November, Q174
- The UK government response to the Science & Technology Committee’s report | Link