Do Respondents Completing Abstract, Hypothetical Priority-Setting Exercises Agree with the Policy Implications of Their Choices?

Stated preference studies are increasingly being used to understand people’s views about the relative importance of different criteria for making health care priority setting decisions. Such studies typically involve presenting survey respondents with priority setting scenarios involving hypothetical patients and medical conditions. In order to minimise bias, researchers usually present the scenarios in an abstract manner and provide limited descriptive information. This has led to concerns that the answers given by respondents may be unreliable. Would the answers have been different if respondents had been better informed about the conditions and the patients affected by them?

 

It is also unclear whether the respondents would agree explicitly with the policy implications of their responses to the abstract choice tasks. The extent to which respondents in social preference studies agree with researchers’ interpretations of their responses has received little attention in the health economics literature to date.

 

A recent OHE study, commissioned by the Pharmaceutical Oncology Initiative, examined the impact of alternative presentations of hypothetical priority setting scenarios, and the extent to which the study respondents agreed with the policy implications of their responses to stated preference tasks. A survey was designed to elicit data on people’s preferences regarding health care priority setting. The questions formed the basis for two focus group discussions and a self-completion Internet survey.

 

The results show that people’s stated preferences regarding hypothetical scenarios are influenced by the way in which the information is presented to them. They also show that people do not always agree with the policy implications of their responses to the stated preference tasks.

 

A common theme arising from the analysis is that it is unwise to make general statements about people’s priority setting preferences based on their responses to very specific choice tasks. Their answers may be driven by specific attributes and parameters described in the choice tasks. For example, respondents might prefer to extend the lives of patients whose life expectancy without treatment is one year rather than those whose life expectancy without treatment is five years. This does not necessarily mean that they would support a general policy of giving priority to patients who are expected to die soon as a result of a medical condition. Their choices might have been different if they had been told that one patient group was much younger than the other, or if the choice had instead been between patients with life expectancies much shorter than one and five years.

 

We found that these kinds of studies are subject to important framing effects. Presenting information about age and quality of life in different ways resulted in different patterns of responses. Researchers often seek to interpret responses to stated preference tasks to draw conclusions about the types of policies that the study respondents would support. However, whether the respondents actually consider these policies to be acceptable or not will depend on whether they have interpreted the concepts underpinning the choice tasks in the same way as the researchers have. Caution is therefore required when using the results of social preference studies to drive public sector decisions, as the results can be sensitive to the methods used.

 

Reference: Shah, K., Chapman, A., Devlin, N. and Barnsley, P., 2015. Do respondents completing abstract, hypothetical priority-setting exercises agree with the policy implications of their choices? Consulting Report. London: Office of Health Economics.

Download the full report here.

 

Related publications include:

Rowen, D., Brazier, J., Mukuria, C., Keetharuth, A., Risa Hole, A., Tsuchiya, A., Whyte, S. and Shackley, P., 2014. Update: Eliciting societal preferences for weighting QALYs according to burden of illness, size of gain and end of life. EEPRU Research Report. Universities of Sheffield and York. [available to download free-of-charge]

 

Shah, K.K., Tsuchiya, A. and Wailoo, A.J., 2014. Valuing health at the end of life: A stated preference discrete choice experiment. Social Science & Medicine, 124, pp.48-56. [available to download free-of-charge]

 

For additional information about this study, please contact Koonal Shah.