Last week, we published a report on using data science in policy. The report covers eight exemplar projects from our new data science team, ranging from improving school and GP inspections to avoiding road traffic collisions. In all of these cases, we found that using machine learning on publicly available data means we can spot struggling institutions sooner, and help them to improve.
The report has spurred a lot of interest, discussion and debate. We believe it’s vital that we make the most of emerging technologies to improve the lives of citizens. But using data science in areas this complex and varied is always going to present challenges. That’s why we also believe that professional decision making must continue to be central to public services.
In part, this is because our current algorithms can know without understanding. Let us say, for an entirely fictitious example, that our algorithm predicted that doctors that prescribe fewer doses of methadone are more likely to be rated as outstanding by the Care Quality Commission (CQC). The model isn’t making a judgement, just a calculation. But what does it actually mean?
From a technical perspective, it means that these factors were statistically important predictors of higher inspection ratings by the CQC in a model conducted on historical data used to train the algorithm, verified against a sample of data that were left out of the training process.
What it certainly doesn’t mean is that there is a causal relationship going on - that a GP will get higher CQC scores if they prescribe less methadone. This kind of data analysis can’t give us this kind of recommendation. At BIT, we spend much of our time running randomised controlled trials to learn what works, and what doesn’t, accompanied by qualitative research to try and find out why that might be. These are often complex, practical tasks taking years - finding a causal relationship simply cannot be done by throwing data at the problem.
This is why it’s essential that data science techniques like the ones we’ve used are combined with professional judgement by human beings. An inspector, a social worker, a teacher, a doctor, a police officer - these people are experts, and if you send them to a GP practice, they can usually tell you not just that it is outstanding, but why it is, and if it isn’t, what might be done to help it improve.
Data science is only going to be useful if we view it as something that can be added to the toolbox that professionals already have available to help them make decisions. It is not a replacement for human beings. That’s why we’re keen to work with those professionals to help find out what kind of tool would be most useful, and work to reduce and avoid as many of the pitfalls that come with using data poorly.
Our report shows that there is huge potential to these techniques to help governments serve people better, but that’s very far from being the end of the story. We need to work to make data science useful, not just powerful. We might not get it right first time, but that doesn’t mean it’s not worth the effort.