Dr Sebastian Rushworth has written today about the serious problem of the underestimation of side-effects in drug trials, which he says should “shake the very foundations of evidence based medicine”.
His article reports on the results of a study recently published in the Lancet Healthy Longevity, funded by the UK Medical Research Council and the Wellcome Trust, which seeks to establish the extent to which drug trials underestimate side-effects by comparing trial data to real world data. The study focuses in particular on blood pressure drugs known as RAAS blockers, which Dr Rushworth explains were chosen because of the number of trials that have been done by different companies. There is no reason the results should not apply equally to other drugs, he says, including Covid vaccines (for which there have been an unprecedented number of adverse event reports despite the trials showing them to be safe).
The results are shocking. The difference was not marginal but out by a factor of three or four.
The real world patients were between 300% and 400% more likely to experience a serious event than the participants in the trials! That is in spite of the fact that the trials, as mentioned above, were using a broader definition of what constituted a serious event. If the trials were representative of reality, then they should have a higher rate of events than is seen in the real world data. Instead they have a rate that is several times lower!
The difference was just as large in trials specifically involving older people, so age differences can’t be the full explanation, he says (though he allows part of it may be that participants in the trials of older people may be healthier and younger than real world patients).
His main explanation, however, is darker. Are the drug companies simply under-reporting adverse events, both in the drug and placebo treatment groups?
The easiest solution, if you want to avoid finding nasty side effects, is to not report them, regardless of which treatment group the participant is in. That will cut down on total adverse events in both groups, which will make any difference between the groups that does exist smaller in absolute terms, and also less likely to reach the level of statistical significance. Voila – the treatment group and the placebo group end up having similar rates of side effects, and the drug company can conclude that the drug is completely safe.
Is that what’s happened here? Are the pharmaceutical companies hiding adverse events? Well, it’s very strange that the real world data shows a rate of serious adverse events that is several times higher than is found in the trials. It’s hard to see how that massive difference could be explained in any other way.
This is a huge problem, he argues, which calls into question the reliability of drug trials for telling us how safe drugs really are.
If the drug trials and the real world data show such wildly different rates of adverse events, then it really begs the question how much we can trust the trials at all. It would be perfectly reasonable in this situation to say that all “evidence” produced by pharmaceutical companies is so suspect that it should be dismissed out of hand, and that only independently funded trials should be used as a basis for medical treatment decisions.
Combined with the routine exclusion of high risk groups from trials, who are both more likely to suffer adverse effects and less likely to get the full benefit, we need to ask how much we really know about the risks and benefits of the drugs and vaccines that we give to people every day.
But what alternative is there, asks Dr Rushworth, when without drugs companies funding trials far fewer would be done at all?
Worth reading in full.
