New: Assessing the Performance of the EQ-VAS in the NHS PROMS Programme

The NHS Patient Reported Outcome Measures (PROMs) programme, introduced in April 2009, is a significant development in the routine collection and use of patient reported outcome data. Currently, data are collected from patients both before and after surgery for four elective surgical procedures in the NHS, with plans to expand the practice.

The NHS Patient Reported Outcome Measures (PROMs) programme, introduced in April 2009, is a significant development in the routine collection and use of patient reported outcome data.  Currently, data are collected from patients both before and after surgery for four elective surgical procedures in the NHS, with plans to expand the practice.  (The procedures are hip and knee replacement, groin hernia repair and varicose vein repair.)

Results from the PROMs data collected already are being used in a broad range of decision making contexts[1].  For example, comparisons of the changes in patient health before and after surgery are being used as one indicator of hospital performance[2].  Commissioners also are using the findings in evaluating the effectiveness and cost effectiveness of a range of services.  Ensuring that the data are as accurate and reliable as possible, then, is crucial.

The EQ-5D, used to collect the PROMs data, has two parts. The first, the EQ-5D profile, asks patients to classify their health based on self-assessed levels of problems (“no”, “some”,” extreme”).  The second is the EQ-VAS, which asks patients to indicate their overall health on a vertical visual analogue scale, ranging from “worst possible” to “best possible” health.  Historically, the EQ-VAS was a warm-up exercise for VAS valuations of EQ-5D, but it has evolved into an integral part of the measure.  This paper focuses on two concerns about the EQ-VAS: (1) difficulties with the data because patients may fail to respond to it as the instructions require and (2) substantial differences between patient responses on the EQ-VAS versus the EQ-5D profile and condition specific instruments.  According to the authors, these issues raise fundamental questions about the role and use of EQ-VAS in the EQ-5D instrument.

In particular, the paper analyses:

1. The different ways in which patients complete the EQ-VAS and how their characteristics affect this
2. How the different ways of completing the EQ-VAS currently are handled in coding the data and the effect this may have on data interpretation and analysis
3. The relationship between the EQ-VAS, the EQ-5D profile and other summary score data in the NHS PROMs programme

Analyses to address the first two points — potential issues with patient responses and how to handle them – are based on matched before-and-after anonymised EQ-VAS responses from 200 patients across all four elective procedures.  The data included background characteristics, namely age, sex, and type of surgery.  The third set of analyses, comparing EQ-VAS data to index weighted measures, were based on patient-level NHS PROMs programme data linked to Hospital Episode Statistics.  The 331,951 anonymised patient records covered all four elective procedures from 1 April 2009 through 28 February 2011.

The authors’ findings suggest ways for improving EQ-VAS by improving both data collection and coding procedures.  They note, for example, that although 95% of patients completed the EQ-VAS in an unambiguous way, fewer than 50% completed it in the way that the instructions intend.  Moreover, the guidance provided by the EuroQol Group on coding the imperfect responses is insufficient.  The authors note that this potentially can result in unnecessary data wastage or variations across users in interpreting and coding that, in turn, may make data less comparable.  “All of these issues,” they note, “could be addressed by providing improved guidance on coding EQ-VAS data or revisiting the instructions for the EQ-VAS”.

The third set of analyses presented in the paper addressed whether and how patient reported outcomes vary using EQ-VAS versus EQ-5D or condition specific instruments.  The authors note that “Concern had emerged from the NHS PROMs programme that the EQ-VAS was not adequately reflecting the health gain for patients resulting from surgery, and was therefore a less useful and appropriate measure of health change than the EQ-5D profile or condition specific instruments.”  Their findings suggest that this is not the case; the relationship between the EQ-VAS and the EQ-5D profile are predictable and consistent. Moreover, some of the differences between the two are attributable to the characteristics of the particular weightings within the EQ-5D index.

Nevertheless, the results of analyses do confirm the observation of PROMs reports that there are clear differences between the EQ-VAS and index weighted EQ-5D and condition specific profiles.  Presumably, patients are freer to consider all components of “health” using the VAS than they are when restricted to “tick boxes” with the EQ-5D profile.  “In essence,” the authors state, ”the EQ-VAS is measuring a broader underlying construct than the EQ-5D profile or the condition specific instruments. This does not mean that the data it produces are less meaningful or useful. Indeed, in applications where the patients’ view of their overall health is the measurement goal, the EQ-VAS is prima facie more appropriate than the use of EQ-5D profile data weighted by general public preferences.”

In closing, the authors note that no discoverable research has been done on how patients or members of the general public interpret the upper and lower endpoints of the EQ-VAS – “best” and “worst” possible health.  Given the importance of the EQ-VAS, they urge that a better understanding be developed of differences in interpretation, the bases for these, and whether they may change with expectations, health or social circumstances.

Download Feng, Y., Parkin, D. and Devlin, N.J. (2012) Assessing the performance of the EQ-VAS in the NHS PROMs Programme. Research Paper 12/01. London: Office of Health Economics.