Sleepio: Lessons from the Frontier of Digital Therapeutics

The National Institute for Health and Care Excellence (NICE) recently issued its first guidance for a digital therapeutic. NICE recommends the use of Sleepio for treating insomnia symptoms in primary care. OHE played a vital role in developing the evidence to support the guidance. In this blog post, Chris Sampson shares the story of how we got there, and some lessons from the experience.

Back in the heady pre-pandemic days of 2018, OHE started working with Oxford Academic Health Science Network (AHSN) on a new project funded through Innovate UK’s Digital Health Technology Catalyst. The project’s overall objective was to address the unmet health care needs of people with insomnia. To do this, a collaborative team would explore options for commissioning Sleepio; a fully automated digital therapeutic that delivers digital cognitive behavioural therapy for insomnia (dCBT-I). OHE was drafted in to consider the cost implications of adopting Sleepio in practice.

The project focused on implementation challenges and did not include a randomised trial. Instead, we designed a quasi-experimental study, looking at primary care costs before and after Sleepio was rolled out. The study was conducted in the Thames Valley region of England, and everybody in the area (with an eligible postcode) could access Sleepio for free between October 2018 and January 2020. We designed an interrupted time series analysis, comparing primary care use before and after the rollout of Sleepio, controlling for the trends in resource use before Sleepio was available. We focussed on how many times people saw their GP and the relevant prescriptions they received each week.

The Oxford AHSN team worked with local primary care practices and a data services company to ensure we could access high-quality anonymised patient data. This was a fundamental part of the project’s success because it allowed the OHE team to focus on developing the most appropriate modelling strategy for a pre-specified data structure. We completed our analysis as planned and published our write-up as a pre-print in medRxiv, making the findings available to decision-makers (including NICE) and other stakeholders as soon as possible. Subsequently, following peer review, the paper was published in BJGP Open.

Starting in late 2020, NICE’s evaluation of Sleepio coincided with planned updates to its evidence standards framework and the digital health technologies pilot. Being part of this learning process meant that the assessment took longer than expected, more than 18 months from start to finish. However, it was rewarding, as researchers, to be part of these developments taking place at the forefront of technology assessment and digital health.

It’s no surprise that Sleepio is the first digital therapeutic to be recommended by NICE. There is a staggering amount of clinical evidence supporting Sleepio; 13 clinical trials and counting. This is an important lesson for other companies developing digital therapeutics; evidence is everything. The documentation provided by NICE for the evaluation – especially the committee discussion – reveals that there was little uncertainty about the effectiveness of Sleepio. This was never likely to be the challenge. The challenge was always going to relate to costs. As a general rule, technologies of this kind, especially digital therapeutics, are expected to be cost-saving. For dCBT-I in the UK, this is not easy to demonstrate. Sleepio has various characteristics that health economists have long recognised as challenging for HTA and economic evaluation.

For starters, the value of web- or app-based digital therapeutics can bear similarities to public health interventions. Many people have sleep problems, and, in principle, the NHS could make Sleepio freely available for anybody to access without engaging with health care professionals. In this case, the benefits of the intervention (and the resource use impacts) are diffuse and are difficult to identify precisely. Another challenge is that effective treatment for sleep problems is lacking (at least in the UK). There are drugs, but these tend to be ineffective, potentially addictive, and even harmful in the long term. NICE guidance recommends CBT for people with insomnia, but few people get it because there aren’t enough therapists in the NHS. Thus, the most appropriate comparator for Sleepio is essentially nothing, or perhaps some low-cost drugs. And it’s difficult to demonstrate cost savings relative to nothing.

Despite these challenges, our study demonstrated that Sleepio rollout was associated with a reduction in primary care costs. This suggests that Sleepio was used as a substitute for GP appointments and prescriptions. Our findings were robust to a variety of specifications, but the non-randomised nature of the study – and particularly the observation of cost differences at the population level – presented a challenge to the NICE committee. Digging into the Internet Archive reveals that NICE postponed the expected publication date of their guidance, and this was in part related to the assessment of this evidence, which called for additional validation procedures.

We’ve learnt a lot from this work, and have witnessed the rapid development of processes for assessing digital therapeutics. Collaboration was key to the success of the research and the delivery of impactful and meaningful evidence for NICE. Real-world data is becoming increasingly important across health care and will be pivotal in evaluating digital therapeutics. Digital interventions provide new opportunities for ongoing data collection from patients and may be safely evaluated in implementation studies, free from the strict conditions of clinical trials. Agencies and public bodies such as NICE must align their real-world evidence and digital health assessment strategies. On a final note, it pays to be patient. An evidence base cannot be created overnight, and novel methods take time to permeate decision-making. Evidence standards for digital technologies are likely to seek cost savings for the foreseeable future; it is never too soon to start generating evidence on the costs of adopting new digital therapeutics.

Citation

Sampson, C., Bell, E., Cole, A., Miller, C.B., Marriott, T., Williams, M. and Rose, J., 2022. Digital cognitive behavioural therapy for insomnia and primary care costs in England: an interrupted time series analysis. BJGP Open. 10.3399/BJGPO.2021.0146.