Science is a well-established way of testing ideas. Scientists have an idea, and then run an experiment to test it. Then they look at the result, to see if they correctly predicted what happened. In many fields of science, including medical science where consequences can be harmful, what happens is so complex it’s impossible to observe everything. To help make sense of it all, scientists use statistics.
Scientists run experiments to test new medicine. They do each experiment many times, recording their results. At the same time, they can run each experiment without the new medicine, instead using a fake treatment called a placebo, and they repeat this lots of times as well.
If the results are very different between the medicine and the placebo, then there’s a good chance that the medicine is causing the difference. But sometimes it might just be a fluke – random chance can cause unexpected results.
This doesn’t tell us how many incorrect results get published. To work out what was going on, US statisticians looked at more than 77 000 published results in five medical journals. They concluded that one in seven published results is likely to be a false positive.
However, science doesn’t end with the first set of published results. Experiments are typically repeated several times, by different laboratories, before the result is accepted. New medicines have to be checked several times before they are tested on humans and several more times before they are available to the general public.
By using multiple trials, in different laboratories, the chance of a medicine being a false positive is extremely low. When a medicine is released for use, doctors still keep an eye on treatments, to make sure they work as predicted. So although individual experiments can be unreliable, science delivers well-tested medicine.