The investigation started within a week of the paper’s publication. It lasted four days and involved replicating the key experiment, with Maddox, Stewart and Randi monitoring every stage and checking for flaws in the procedure. They observed the handling of several test tubes containing basophil blood cells, some of which were treated with the homeopathic allergen solution, while the rest were treated with plain water and acted as a control. The task of analysing the test tubes was given to Elisabeth Davenas, Benveniste’s assistant, and yet again the result was the same as it had been for the last two years. More of the homeopathically treated cells showed an allergic response than the control cells, implying that the homeopathic solution had genuinely triggered a reaction in the blood cells. Even though the homeopathic solution no longer contained any allergen, its ‘memory’ of the allergen seemed to be having an impact. The experiment had been successfully replicated.
The investigators, however, were still not convinced. When Davenas analysed the test tubes she knew exactly which ones had been treated with the homeopathic solution, so the investigators were concerned that her analysis might have been deliberately or unconsciously biased. In Chapter 2 we discussed the issue of blinding, which means that patients in a trial should not be aware of whether they are receiving the real treatment or the placebo control treatment. Blinding is equally applicable to doctors and scientists. They should not be aware of whether they are administering or studying the real or the control treatment. The aim of blinding is to minimize bias, and to avoid anybody being influenced by their expectations.
Consequently, the Nature team requested Davenas to repeat the analysis, but only after they had blinded her to the contents of the test tubes. Maddox, Randi and Stewart went into a separate room, blanked out the windows with newspapers, removed the labels from the test tubes and replaced them with secret codes that they would later use to identify which samples had been treated with homeopathic solution and which had been treated with water. Davenas repeated her analysis, while colleagues from around the laboratory gathered to await the final result. The Amazing Randi, as he is known on stage, amused the crowd with a few card tricks to help ease the tension.
Eventually Davenas completed her analysis. The secret codes were revealed and the Nature team identified which test tubes had been treated homeopathically. This time the results showed that the basophils in the homeopathically treated samples had not reacted differently from the control basophils treated with plain water. The experiment had failed to demonstrate the sort of effects that Benveniste had been finding for the last two years. The results showed no evidence to support homeopathy, and instead they were in line with conventional scientific thinking and all the known laws of physics, chemistry and biology. Some of Benveniste’s colleagues burst into tears at the announcement.
Subsequently it emerged that Benveniste had never personally conducted any of the experiments, but had always left everything to Davenas. Moreover, she had always conducted the analysis in an unblinded manner. This meant that it was highly likely that she had accidentally and consistently introduced biases into the results, particularly as she herself was already a strong believer in the power of homeopathy and was keen to prove its efficacy.
When Nature published the results of its investigation, the journal pointed out several problems with Benveniste’s approach to research. These criticisms included statements such as: ‘We believe that experimental data have been uncritically assessed and their imperfections inadequately reported.’ Moreover, the journal highlighted the fact that two of the researchers who had contributed to Benveniste’s original paper had been partially funded by a French homeopathic company with an annual turnover of over 100 million. Corporate funding is not necessarily problematic, but such potential conflicts of interest had not been formally disclosed. Despite these criticisms, the Nature investigators were keen to stress that they were not accusing Benveniste of deliberate fraud, but merely that he and his team were deluding themselves and had not been conducting their experiments rigorously.
A lack of rigour, particularly a lack of blinding, can seriously bias any scientific result even for the most honest and well‑intentioned scientist. Imagine the following scenario: a scientist has staked his reputation on the hypothesis that men have superior spatial awareness and motor skills, and he thinks he can demonstrate that this is the case by inviting men and women to draw freehand circles and then comparing the quality of their drawings. The experiment begins–the men and women draw their circles, they write their names at the top of the papers, the drawings are collected by an assistant and handed to the scientist, who judges the circles by eye and gives each one marks out of ten. However, because he can see the names of the artists at the top of each drawing, he might be tempted subconsciously to mark the men’s circles more generously. Consequently, regardless of the truth, it is more likely that the resulting data would support his hypothesis that men are better than women at drawing circles. By contrast, if the experiment were to be repeated and the artists were given numbers to disguise their gender temporarily, then the prejudiced scientist becomes blinded and is more likely to give a fairer assessment of each circle. The new result is likely to be more reliable.
In the Benveniste case, the problem was that Davenas was unblinded and prejudiced in favour of homeopathy, and this combination of factors could have biased her results. In particular, Davenas’s experiments required her to judge whether a homeopathic preparation caused blood cells to exhibit an allergic reaction, which is not a clear‑cut decision, even when the cells are viewed through a microscope. Indeed, judging the extent of a cell’s allergic reaction is similar to examining a circle’s roundness: both are equally prone to personal interpretation and bias.
For example, Davenas would have come across many borderline cases–has the cell undergone an allergic reaction or not? There might have been a subconscious temptation to judge such borderline cells as exhibiting allergic reactions if she knew that they had been treated homeopathically. Or she might have been subconsciously tempted to give the opposite judgement if she knew that they had been treated with plain water. However, by asking Davenas to repeat the experiment without any labels on the test tubes, the Nature investigators ensured that she was blind and unbiased in her decisions; whereupon the homeopathic solutions and water led to similar results. A fair test had shown that the homeopathic solutions had no impact on the basophil cells.
While Benveniste readily accepted some elements of the criticism, he steadfastly defended the core of his research and argued that the results that he had accumulated over the course of two years could not be negated by what the Nature team had observed in just a few days. He explained that the mistakes that had been witnessed by Maddox, Randi and Stewart were caused by the unusual circumstances, namely that his team was working under intense pressure and in the media spotlight.