Analysing Hospital Variation at the Level of EQ-5D Dimensions

Measuring and publishing hospital data on achieved health outcomes is becoming routine in many health care systems. League tables of hospital quality – based on post-surgical survival rates, for example – are used to highlight variation in performance across providers. These measures, however, reveal little about the health of the vast majority of patients and may fail to detect important variations in quality.

Prof Devlin

Measuring and publishing hospital data on achieved health outcomes is becoming routine in many health care systems.  League tables of hospital quality – based on post-surgical survival rates, for example – are used to highlight variation in performance across providers. These measures, however, reveal little about the health of the vast majority of patients and may fail to detect important variations in quality. More comprehensive measures of patients’ health outcomes are needed as the basis for a better assessment of hospital performance and identification of best practice.

In this paper, OHE’s Nancy Devlin collaborated with colleagues from the Centre of Health Economics at the University of York to measure variation in hospital quality using a new, routinely collected dataset on patient‐reported outcome measures (PROMs). Since April 2009, all providers of publicly‐funded inpatient care in the English NHS have been required to collect such measures for four elective procedures: unilateral hip and knee replacements, varicose vein surgery, and groin hernia repairs. Patients undergoing these procedures are asked to complete the EQ-5D, which produces generic measures of health, as well as a condition-specific questionnaire both before and several months after surgery.

The use of PROMs in the context of routine performance assessment on a national scale is new; it requires an appropriate methodology that takes into account both the characteristics of the data and their intended use as measures of relative quality of hospital treatment. The NHS Information Centre (IC) has developed a preliminary risk-adjustment methodology that currently is applied to the PROMs data. In this study, the authors build on these efforts and propose the following two refinements.

First, the authors argue that the data should be analysed at the level of PROM-item responses instead of summarizing them as a single score. Collapsing EQ‐5D health profile data into a single value by means of weighting loses some information on the underlying nature of the patient’s health and can bias statistical inference. It also raises questions about whose views the scores should reflect – those of the general public (as is generally the case when EQ-5D data are used in economic evaluation) or those of patients.

Second, the authors note that patients’ health outcomes are likely to be influenced by both observed factors (e.g., age and gender) and unobserved factors (e.g., differences in the way patients report their health, unobserved medical conditions) that are outside the hospitals’ control. These factors need to be taken into account to allow for a fair comparison of hospital quality. The authors argue that the risk‐adjustment methodology should distinguish random variation and differences across patients from systematic variation in quality among providers. Because PROMs data are available both at the patient level and at the provider level, multi-level models are a good way of exploring this.

This study focuses on hip replacement patients and studies hospital variation in changes in self-reported health on the five dimensions of the EQ-5D: mobility, self-care, usual activities, pain & discomfort, and anxiety & depression. The results suggest that patients undergoing hip replacement surgery report, on average, improvements in health along each of the five dimensions six months after surgery. However, hospitals vary in their systematic impact on post-treatment health. This is especially pronounced on dimensions related to functioning (i.e., mobility, usual activities) where several performance outliers can be identified. In contrast, pain relief, one of the primary goals of hip replacement surgery, is achieved by all hospitals to a similar extent. Given that the EQ-5D utility index attaches high weights to the pain dimension, but relatively low weights to mobility and usual activities, the authors demonstrate how quality assessment using summary scores can provide an incomplete, potentially misleading assessment of hospital performance.

This study is the first to analyse hospital performance in terms of the EQ-5D dimensions and to use multilevel modelling methods to do so. It demonstrates how the data being generated from the NHS PROMs programme can be used in creative ways to generate completely new insights into hospital performance and quality. Previously, PROMs data were available to download only in an aggregated form, for example, as averages for each hospital. Now, patient level data, including patients’ self-reported health on the dimensions of the EQ-5D before and after surgery, are available for download from the NHS IC.

This research was undertaken as part of an ongoing research programme funded by an NIHR Health Services Research Grant.

Download Gutacker, N., Bojke, C., Daidone, S., Devlin, N., and Street, A. (2012) Analysing hospital variations in health outcomes at the level of EQ-5D dimensions. CHE research paper. 74. York: Centre for Health Economics, University of York.

For a related paper by this research team, download
Gutacker, N., Bojke, C., Daidone, S., Devlin, N., Parkin, D. and Street, A. (2011) Truly inefficient or providing better quality of care? Analysing the relationship between risk-adjusted hospital costs and patients’ health outcomes. CHE Research Paper. 68. York: Centre for Health Economics, University of York.

For more on PROMs, download Devlin, N. and Appleby, J. (2010) Getting the most out of PROMs: Putting health outcomes at the heart of NHS decision making. London: King’s Fund and Office of Health Economics.

See also Parkin, D. and Devlin, N. (2011) Using health status to measure NHS performance: Casting light in dark places. BMJ Quality and Safety. Online first. 21 September. [Available for download by subscribers only]