NICE Decisions: Cost-effectiveness is the Most Significant Predictor

Using data for 1999-2011, this thorough analysis explores what has most influenced NICE’s decisions and whether this has changed.

Established in 1999, the National Institute for Health and Care Excellence (NICE) undertakes appraisals of selected technologies and issues guidance intended to ensure quality and value for money. Its decisions are binding within the NHS and also affect decisions by health technology assessment bodies and payers in other countries.

Established in 1999, the National Institute for Health and Care Excellence (NICE) undertakes appraisals of selected technologies and issues guidance intended to ensure quality and value for money. Its decisions are binding within the NHS and also affect decisions by health technology assessment bodies and payers in other countries. Understanding what factors actually affect NICE decisions, then, is important.

Just released is a new OHE Research Paper that explores which factors have most influenced NICE decisions through the end of 2011. The authors are Helen Dakin (Oxford University), Nancy Devlin, Yan Feng and Phill O’Neill (OHE), Nigel Rice (University of York) and David Parkin (King’s College London).

The research has two objectives: identifying the influence of cost-effectiveness and other factors on NICE decisions, and investigating whether NICE’s decision making has changed over time. NICE’s decisions were modelled as binary choices, i.e. “yes” or “no” recommendations about use of a health care technology in a defined patient population. Independent variables included clinical and economic evidence about the technology; the characteristics of the patients, disease or treatment; and “contextual factors affecting the conduct of health technology appraisal”. Data on all NICE decisions published by December 2011 were obtained from HTAinSite.

The authors note that “Cost-effectiveness alone correctly predicted 82% of decisions; few other variables were significant and alternative model specifications led to very small variations in model performance”. The probably of rejection increased significantly with increases in the incremental cost-effectiveness ration (ICER), with the tipping point most often being between £39,000 and £44,000. No significant change in this was observable over time.

Specific diseases were the only other significant influence. The study notes that “NICE rejections were significantly less likely for cancer and musculoskeletal disease, but significantly more likely for respiratory disease”, although cause and effect were unclear. The authors point out, however, that other one-off factors might have been considered by NICE, but not referred to in the official written recommendation. In addition, several factors that NICE has said do influence its decisions would be difficult to measure for the purposes of this research—e.g. disease severity and “innovation”. Also not included were measures of the robustness of clinical evidence or the extent of clinical uncertainty; both, the authors note, still have not been “properly captured . . . conceptually and empirically.”

Download Dakin, H., Devlin, N., Feng, Y., Rice, N., O’Neill, P. and Parkin, D. The influence of cost-effectiveness and other factors on NICE decisions. Research Paper 13/06. London: Office of Health Economics.

See our earlier post for a summary of an OHE seminar on this research in a slide presentation.

For addition information, please contact Professor Nancy Devlin.