Last week a paper was published in the Lancet, looking at complications of patients admitted with Covid between January 17th and August 4th 2020.
The publication was followed by extensive press coverage, most of which attributed findings to the paper that weren’t supported by the data. It will come as no surprise to readers that the press reaction featured dire warnings of imminent catastrophe as hospitals are ‘once again’ overwhelmed by Covid patients and serious risks to the long-term health of young people from viral infection.
Accordingly, I have examined the paper and summarised my initial findings for readers. For clarity and avoidance of doubt, any medical study contains flaws and biases. Some of these flaws relate to data collection or measurement, some relate to data analysis. The majority of bias is found in the way data is presented and interpreted. When assessing papers for publication, reviewers are supposed to pick up on such biases and (as far as possible) screen them out prior to the paper being accepted. I should also say that reviewers themselves are subject to their own biases and I’m no exception to that rule.
Before I get into the weeds, I should explain what a ‘complication’ means in medical terms. Essentially, a ‘complication’ is an extra problem that occurs during the management of the primary disease process. It’s something that complicates the routine management of the underlying condition. So, for example in surgery, if a patient has a gallbladder removed, most of the time that goes very well and the patient will go home the next day (or even on the same day). However, if the patient develops a chest infection after the operation, this is a ‘complication’. It means the patient may need to stay in hospital longer, have more drugs prescribed, have extra treatment like chest physiotherapy and so on. In bad complications (for example a heart attack, or a serious deep vein thrombosis), there may be ongoing problems caused by the complication. Some patients are at particularly high risk of specific complications – for example in the case of the gallbladder and the chest infection, an obese heavy smoker is much more likely to suffer the complication than a normal weight non-smoker.
The first thing to note is that this is a big study – 80,388 patient records collected prospectively and entered by specific specialist nurses and medical students in over 300 hospitals. There was a missing data percentage of under 10% across the piece (not bad). So, barring inaccurate data entry, this is quite a robust data collection methodology – certainly better than retrospectively trawling through hospital notes.
The definition of ‘complications’ was quite wide, especially in the respiratory and renal domains. For example, the requirement to ventilate the patient in a prone position was regarded as a ‘complication’ – in my view, that is just a marker of disease severity, not a complication like super added bacterial pneumonia. Similarly, the definition of ‘renal complication’ was a 1.5 times rise in serum creatinine levels – personally I don’t buy this as a complication. Such elevations are not uncommon in acute disease states – I would have preferred the definition of ‘renal complication’ to have been restricted to patients requiring external renal support by hemofiltration or hemodialysis. While it might fall within the WHO dictionary definition of a renal complication, I’m not convinced the severity criteria have been properly set in this study. If you set the parameter too wide, you can bias the study by including ‘complications’ that aren’t actually complications – just routine manifestations of the disease process and this can skew the conclusions drawn from the data.
The headline figures were that the overall mortality rate was 31% and the complication rate 49.7%. Of the surviving patients, 43% had at least one complication. I was interested to see that only 85% of the cases assessed had a positive PCR test for COVID 19. The authors explain this by writing that the patients had disease states that clinically looked like Covid. Personally, I would not have passed this paper unless all the patients entered into the study had positive tests – a negative test rate of 15% is quite high and implies that the results are corrupted by patients who never had Covid in the first place. I’m really quite surprised by that – it would be easy to eliminate the 15% and still have a very large study. The rate of ‘complications’ seems high to me, and that might be due to excessively wide parameters. As Covid, is a multisystem inflammatory condition, it’s not surprising that there is widespread organ involvement – I wouldn’t necessarily define minor dysfunctions as complications, more an expected part of the disease process.
The headline findings are not that surprising. In summary, older patients with more pre-existing comorbidities tended to have more complications and a higher mortality. Younger patients tended to have fewer pre-existing conditions and fewer complications. Males had more complications than females. Surviving patients had a lower incidence of complications (45%) than patients who died (62%). Cardiovascular or respiratory complications were more likely to be associated with death than other organ system problems. Complications were more common in patients requiring critical care. Obese patients were more likely to have renal and respiratory complications. All that is very well known from already published studies. I’m not going to test readers patience by going into all the quantitative details.
My main criticism of this paper lies in the conclusions drawn from the data. The majority of the discussion section concentrates on the incidence of complications from acute Covid in hospitalised younger patients – defined as under 50 years of age. Yet only 12.6% of the sample size were under 50 years of age. Very little discussion is made of the other 87.4% of patients in the older age groups, who actually had the worst outcomes.
Major emphasis is laid on the incidence of renal complications seen in the younger subgroup and what implications that might have for the future health of the patients. Inference is drawn that such acute kidney injury may lead to higher risks of subsequent renal failure and heart disease in later life.
Yet the authors extrapolate those conclusions based on citations of other papers which do not reflect the subgroup of younger patients referred to in their own figures. In my view it is not reasonable to compare a group of patients under 50 experiencing transient acute kidney injury in the context of another acute disease with a cohort of much older patients having AKI after recent heart attacks (as in one of their citations). Equating the long-term outcomes from these two distinct groups is likely to be a flawed assumption.
I note with interest that the incidence of acute kidney injury as a proportion of overall complications in each age group decile up to the over 90s was remarkably consistent at between 32% and 35%. No distinction was made in the analysis between people requiring renal replacement with dialysis or filtration and those experiencing transient biochemical renal dysfunction that was correctable with intravenous fluid replacement and other simple interventions. This observation supports my suspicion that the parameters of ‘renal injury’ have been set too wide to distinguish between mild dysfunction of no long-term consequence and serious renal damage.
I was also surprised by some of the data findings not commented on in the discussions. Examples include the fact that 40% of patients did not have a data entry in relation to their smoking status (never smoked, current smokers or ex-smokers). This is a routine question in every hospital admission, so such a high level of data dropout, particularly in a predominantly respiratory infection, requires some explanation.
Another example is in table 2 where it suggests that 24.7% (17,652) of the admitted patients did not require any oxygen. To me this rather begs the question of why they required hospital admission in the first place. Is it possible that these 17,652 patients were in hospital for some other reason and just happened to have a positive Covid test by co-incidence? That is an important question which needs addressing in general terms, not just in this paper – and this fairly comprehensive data set might shed light on the matter. I wonder why the authors didn’t mention it. Had I been reviewing this paper I would certainly have asked them revise the manuscript to add a comment here.
Further, the table suggests that 60,035 patients (84.7%) did not require ‘non invasive ventilation’. This seems like an improbably high figure to me. Once again, as with ‘renal injury’, definitions are important – it may be that oxygen by face mask was not included under ‘non invasive ventilation’ and that this definition was reserved only for CPAP masks or higher. But still, it’s important for these definitions to be made clear to the reader.
In summary, this is an interesting and worthwhile study which has great depth and requires detailed assessment – I have done a preliminary analysis to provide a quick opinion and I might return to torture readers with further analysis at a later date. There is only so much studying one can cope with on a sunny Sunday afternoon!
My overall impression is that the authors have concentrated on a tiny fraction of the information and largely ignored much of their own data. This maybe because they intend to publish further work looking at different aspects of the database in the coming months – so called ‘salami slicing’ is common in scientific publications. The more papers one can get out of the same database, the greater the academic heft and prestige.
It is not uncommon in scientific literature for groups to adopt ‘an angle’ designed to hook the target journal into publishing their article based on the likely reaction of the scientific community in relation to citations. Citations drive the ‘impact factor’ of the journal, and this is basically the currency of academia – the higher the citation rate, the greater the prestige, and the more likely one is to attract further grants from funding bodies.
So, there may well be a reason why the authors chose to focus so heavily on the incidence of complications in the small subset of the under 50s. Nevertheless, I think they have over-egged the extrapolation into long term complications by using non comparable reference groups. It’s a curious co-incidence that the familiar ‘talking heads’ have also been pushing the narrative around ‘long Covid’ in the young and the risk to younger people of even mild infection with Covid. I may be revealing my own biases by suggesting the discussion section of this paper was written more to attract press attention than scientific citations.
If I had been asked to review this paper, I would not have passed it for publication in its current form without extensive revision of the discussion section – but then reviewers are often selected based on the biases of the journal editors rather than on strict scientific objectivity. My apologies to readers for drawing back the curtain on the reality of medical/scientific publications and revealing some of the less obvious biases in the system . When you hear politicians chanting the phrase ‘following the science’, it is important to understand how ‘the science’ works and how it is presented to the technical and the lay audience.
It’s one of the many reasons why consistent scepticism is so important. As the Nobel prize winning physicist Richard Feynman wrote, “Science is the belief in the ignorance of experts.”
The author is a senior doctor, formerly with the NHS, who contributes regularly to the Daily Sceptic.
