Today we’re publishing a piece by James Lewisohn, who says the problem with complex predictive models is they’re too unreliable to be trusted, but he’s not convinced they’re reliably unreliable enough to be banned.
What are the world’s worst inventions? Winston Churchill famously regretted the human race ever learned to fly. I don’t (I’m looking forward to my next holiday too much). Instead, observing the destruction wrought by government pandemic responses predicated upon projected Covid cases, I’m beginning to regret mankind ever invented the computer model.
I have form here. I spent the early years of my career building financial models, hunched over antique versions of Excel on PCs so slow the software might take twenty minutes to iterate to its results – which, once received, were often patently wrong. I developed a healthy mistrust for models, which frequently suffer from flaws of design, variable selection, and data entry (“Variables won’t. Constants aren’t,” as the saying goes).
Models allow outcomes to be presented as ranges. In business, it’s often the best-case outcome which kills you – early-stage companies typically model ‘hockey stick’ revenue growth projections which mostly aren’t realised, to the detriment of their investors. In pandemics, though, beware the worst case. In December, SAGE predicted that Covid deaths could peak at up to 6,000 a day if the Government refused to enact measures beyond Plan B. The actual number of Covid deaths last Saturday: 262.
SAGE’s prediction was its worst-case analysis, but the fact that the media (and then the Government) tends to seize upon the worst-case is nothing new. In 2009, Professor Neil Ferguson of Imperial College, to his subsequent regret, published a worst-case scenario of 65,000 human deaths from that year’s swine flu outbreak (actual deaths: 457). As Michael Simmons noted in the Spectator recently: “The error margin of pandemic modelling is monstrous because there are so many variables, any one of which could skew the picture. Indefensibly, Sage members are under no obligation to publish the code for their models, making scrutiny harder and error-correction less likely.”
Worth reading in full.
To join in with the discussion please make a donation to The Daily Sceptic.
Profanity and abuse will be removed and may lead to a permanent ban.