Test and Trace has been accused of relying on “dodgy” statistics to justify its £22bn budget after it emerged the service is claiming credit for anyone who isolates with Covid.
Labour hit out as a string of statisticians and academics questioned claims by the controversial service that it was having a significant impact on the R number that measures the spread of the virus.
Test and Trace chair Dido Harding repeated this week to MPs her claim that it had cut the R number by up to 0.6 and was on track to cut it by 0.8 in high prevalence areas of the UK.
But it has emerged that the boast is calculated by comparing the service’s impact with what would happen if no one self-isolated upon getting symptoms of the virus.
Moreover, details buried in Test and Trace’s own business plan reveal that 90% of its claimed cut in transmission comes from people who isolate as soon as they get symptoms – even before coming into contact with the service.
The remaining 10% comes from those who isolate after being told they have a positive test.
A footnote states: “The structure of the model means that, if we were to assume
people would isolate on the basis of symptoms without a test, then a large proportion of the modelled transmission reduction could be achieved from the public guidance to isolate after symptom onset alone.”
Shadow health minister Justin Madders said that the admission appeared to confirm the minimal impact Test and Trace was having on cutting the spread of the virus.
“Test and Trace appears to be relying on dodgy statistics to justify its colossal £22bn budget,” he told HuffPost UK.
“The claims made about the potential impact of Test and Trace are not supported by published data and it seems quite remarkable that it is based on the assumption that there would be no self-isolation taking place at all without their involvement.
“£22bn is a huge sum of money but for a fraction of that we could have a much more effective system of controlling the virus by paying for proper financial support and better public information for those who have the virus.”
The government’s own Sage committee of scientific advisers warned last October that Test and Trace was “having a marginal impact on transmission” and urged more support for people to quarantine at home.
Harding was upbraided by Commons science and technology committee chair Greg Clark this week for failing to publish the full statistical model that she was relying on to make her claims about cutting R.
“In this committee, the science committee of the House of Commons, the practice of science is to be transparent, to publish workings so that people can verify them and can scrutinise them. I see no reason why that should be withheld from the committee and indeed the public,” he said.
Asked whether she had been “misleading” Parliament with the claims, Harding committed to publishing the technical detail of the model within the next fortnight.
“We are not quality-assuring the calculation, we’re quality-assuring the technical description of what we’re doing,” she said.
“These are complex technical calculations. And we’re extremely mindful that it’s important that not only is the calculation correct, which we are confident in and are not changing, but the explanation of the analysis that is being conducted is easy to understand, and digestible and helpful.”
But the committee wrote on Friday to Harding to summon her back this month to explain the calculation. Test and Trace promised in November its “technical annexe” would be “published shortly”.
One NHS source described the R impact calculation as “mad”, while a senior academic said it was “just bollocks”.
Leading statisticians now question both the methodology and the lack of any published evidence to support Test and Trace’s claims.
Professor Sir David Spiegelhalter, chair of the Winton Centre for Risk and Evidence Communication at Cambridge, said: “It is wholly inappropriate to make claims about the impact of Test and Trace by comparing it with the situation in which people do not self-isolate.
“The reasonable comparison is between a centralised tracing service, and a simpler strategy in which people are given strong advice to self-isolate and to inform their household and any contacts. Then we could see whether these funds could be better spent, in a way that could prevent more infections.”
Professor Jon Deeks, who leads Birmingham University’s biostatistics department, said: “This is just complicated modelling and predicting and hypothesising based on all sorts of assumptions. It’s not a real measurement of what it’s doing. The key measurements are the number of people who get the test results within 24 hours and the number of their contacts who are traced within the next 24 hours.”
Stian Westlake, the chief executive of the Royal Statistical Society, said: “The government needs to be clear and rigorous about how it reports the evidence for its testing programmes if it is to maintain public trust. Since last summer, the RSS has been calling for better data reporting from the Test and Trace programme, and is still waiting.
“R gives us an idea of how much more we need to achieve to bring a growing trend under control or to prevent an under-control situation spiralling out of control, so it is important that it is calculated, measured and reported with care.”
Dr Alex Crozier, of UCL’s division of biosciences, said: ”We of course first need to ensure that Test and Trace is comparing itself to a meaningful and appropriate scenario – comparing itself to a scenario where no one isolates even with symptoms does not really tell us anything about the impact of Test and Trace.
“Of course, as Dido Harding herself has actually recognised now at numerous select committee hearings, the key limiting factor on the impact of Test and Trace, and the pandemic response in general, is the lack of support to self-isolate.
“The usefulness of testing is determined by the effectiveness of the intervention a positive result triggers. Given the barriers to self-isolation faced by those at highest risk of a positive result, how can we justify all the money and resources poured into testing, if the government is not providing proper support to isolate?”
Kate Lamble, of BBC Radio 4’s More Or Less statistics programme – which analysed the numbers this week – said: “Analysing Test and Trace’s performance by comparing it to a situation where no one self isolated at all is at the very least a bit odd.
“Especially when back in March, when there weren’t many tests available, the advice was to self isolate if you experienced any symptoms.”
Test and Trace insists its model is “externally reviewed” but has so far not revealed who has done the review.
A Department of Health and Social Care spokesperson said: “We have always been clear that people must self-isolate when they have symptoms or are contacted by NHS Test and Trace – this is crucial in breaking chains of transmission and reducing the R number.
“NHS Test and Trace has reached over seven million positive cases and their close contacts, with 93.2% of all contacts told to self-isolate in the latest week of data.
“With a more transmissible variant in circulation, we are at one of the toughest moments in the pandemic. It is vital, even once vaccinated, people continue to abide by the rules and that means staying at home, and putting hands, face, space into practice every day.”