Estimating a cost-effectiveness threshold reflecting the opportunity cost of adopting a new technology in a health system is not easy. This OHE research paper provides empirical evidence on the relationship between health outcomes and health expenditures in England. Results suggest that setting a cost-effectiveness criterion for NICE may not be capable of being synthesised using scientific methods alone, but involve political judgements.

Estimating a cost-effectiveness threshold reflecting the opportunity cost of adopting a new technology in a health system is not easy. This OHE research paper provides empirical evidence on the relationship between health outcomes and health expenditures in England.  Results suggest that setting a cost-effectiveness criterion for NICE may not be capable of being synthesised using scientific methods alone, but involve political judgements.

Two methods are used to explore the marginal relationship between health expenditure and health outcomes: Data Envelopment Analysis (DEA) and Quantile Regression (QR). DEA allows the incorporation of multiple outcomes (not just mortality) and the measurement of efficiency and scale elasticity, while QR allows us to look for non-linearities in the relationship between spending and mortality. 

DEA was applied to health outcomes and health expenditure data from 151 Primary Care Trusts (PCTs) (now Clinical Commissioning Groups) in England across seven clinical areas termed Programme Budget Categories (PBCs). Two environmental variables were selected (the deprivation index and budget shortfalls against formula) to adjust for factors affecting efficiency that were outside of the control of PCT managers. The QR method was applied to estimate the mortality rate as a function of health expenditure and a set of covariates using data from 151 PCTs in England across six of the PBCs.  The method recognises the non-negative, highly asymmetric and leptokurtic distribution of health expenditure. Point estimates of the mortality elasticity to health expenditure are compared at different parts of the mortality distribution.  Finally, we compare the ranking of PCTs according to the DEA efficiency scores and the outcome elasticities estimated in the QR approach.

Results from DEA show that efficiency varies across PCTs and PBCs. PCTs achieve a range of health outcomes which cannot be adequately explained by concentrating on reductions in the mortality rate. The results from QR analysis provide evidence of heterogeneity across PCTs and PBCs regarding the way health resources are used to improve outcomes. The results suggest that the marginal effect of health expenditure on the mortality rate is not constant across PCTs and PBCs.

The comparison of PCT rankings from the DEA and QR analyses are consistent and robust. In general, efficient PCTs (based on the DEA results) tend to have a lower absolute value of mortality elasticity to health expenditure (based on the QR results). A plausible explanation for these results is that PCTs operating efficiently in a PBC tend to have lower rates of mortality, and for most disease areas, the lower the mortality, the harder it is to achieve additional reductions. 

Estimation of an opportunity cost-based cost-effectiveness threshold using a health production function approach involves many assumptions about the behaviour of the implied function.  These are compounded by the nature of the programme budgeting data that are used for estimation. This study uncovers further problems with assumptions that may underpin attempts to obtain a simple singular system-wide threshold estimate. 

This study provides empirical evidence of production inefficiency, that is the inability of some PCTs to achieve the best practice performance found in others. This means that estimates of the opportunity cost of introducing new technologies based on average performance could be (i) biased and (ii) subject to far greater variation than normally assumed.  Moreover, the PCTs found to be inefficient vary between PBCs, confounding further the plausibility of estimates based on averages.  There is evidence for some PBCs that some apparent inefficiencies result from adoption of a different underlying production function technology, casting further doubts on the assumption of a common production function for all, that underlies a common threshold.

One way to approach this problem is to accept that there are multiple sources of information relevant to the setting of cost-effectiveness criteria to be used across the NHS and that these may not be capable of being synthesised using scientific methods alone.

The research was funded by the Association of the British Pharmaceutical Industry (ABPI).