Due to the Unprecedented Nature of H.R. 3, the CBO Analysis is Unreliable and Can’t Inform Policymaking

As we have discussed in the previous blogs in our Series on H.R. 3, the Congressional Budget Office (CBO) have attempted to estimate the potential effect of H.R. 3 on pharmaceutical research and development and biopharmaceutical innovation. In this blog we discuss is detail why the assumptions in the CBO’s model are too uncertain to be useful for policymakers.  

A Recap of the CBO’s Analysis

The CBO estimated that H.R. 3 would reduce direct Federal spending by nearly $500 billion between 2023 and 2029. According to their original analysis, “negotiations” are expected to reduce prices of selected drugs by 57-75% relative to current prices, which translates to a reduction in US revenue of 34-44% across the Medicare and commercial insurance markets. The analysis estimates a 19% fall in global revenues accounting for projected price rises outside of the US following the introduction of international reference pricing under H.R. 3 within the US market.

Despite H.R. 3’s unprecedented impact on the world’s largest pharmaceutical market, the CBO model projected that only 8 fewer new medicines (as measured by the number of New Molecular Entities (NMEs) approved by the Food & Drug Administration (FDA)) would be developed over the first decade of the policy, with up to 30 further new medicines lost over the subsequent 10 years. This is equivalent to a 2.7% reduction in annual drug approvals in the first decade and a 10% reduction in the second decade.

The CBO Have Attempted a Noble But Impossible Task

Unfortunately, it is likely that policymakers will be misled by the findings of the CBO report when weighing the cost (lost health innovation) against the benefit (reduced Federal spending). Leading experts participating in our expert elicitation exercise concurred, with six of seven experts agreeing with the statement ” policymakers have not been provided with a sufficiently accurate estimate of the cost of the policy to adequately balance the risk of such a decision”. Given the magnitude of the true impact of H.R. 3 could be much greater than the CBO’s findings account for, it is crucial that the limitations of the CBO’s analysis are recognised.

In this blog, we explore the sources and extent of the substantial uncertainty surrounding the potential impact of H.R.3 on pharmaceutical innovation and, ultimately, on patient and population health.

The CBO Baseline is Wrong: The Pace of Innovation is Changing

The CBO’s estimate of the impact of innovation is forecast from a baseline of innovation as measured by the number of new drugs approved by the FDA in the absence of the policy. The CBO use 30 per annum but the actual number of new drugs (NMEs plus new therapeutic biologics approved by CDER) over the past 10 years has ranged from 22 to 59, averaging approximately 40. The CBO seem to acknowledge this issue as their recent simulation model assumes an increased baseline of 44 NMEs a year. However, their baseline assumption remains constant despite a change in the rate of NMEs approved. In the last five years, there has also been an upward trend in the number of NMEs per annum.  One expert commented:

There was disagreement among experts on whether this trend would continue. There was some consensus that the number of new NMEs launched was likely to be higher in the next decade than the previous decade due to maturation of technologies like CRISPR and AI. The disagreement stemmed from differing views of the reaction of the developers of orphan drugs. Some argued that, given the trend of reduced patient populations targeted by each NME due to technological progress in disease characterisation (e.g., genomics), a further decrease in their expected revenue due to the H.R. 3 policy might lead to a displacement of investment in them.  Overall, the median baseline number of NMEs estimated by the experts was 45- 50% higher than the CBO. 

Uncertainty around this baseline level of innovation is important as it changes the CBO’s projection of overall impact significantly. All other assumptions equal, changing the baseline assumption of 30 to 40 would lead to 40 fewer drugs launched in the decade 2030-39 (33% greater impact), and increasing it to 50 would lead to 50 fewer drugs launched (67% greater impact).

Historic Studies Can’t Shed Light on Such an Unprecedented Change

OHE reviewed and summarised the four key papers that informed the CBO analysis. That review concluded that, whilst each of the papers have their merits, none provide reliable insights for a policy change as significant as H.R. 3 due to differences in the scale of impact, market size, existing price regulation, or geography.

In particular, the CBO relies on estimates of the elasticity of innovation to market size- i.e. the relationship between revenue and innovation- derived from data on smaller, price-regulated markets where innovation is less responsive to market size than in the dominant US market. The estimates are also from a period when venture capital was not nearly as mobile as today. Together these factors suggest that the elasticity of innovation could be larger than previous academic studies.

As part of our expert elicitation study, experts from both academia and industry were asked to provide their own estimates of the CBO model parameters with the aim of trying to achieve consensus among the group. The experts, who in this exercise did not represent the investment or venture capital world, were divided on whether the estimates of the elasticity of innovation from the literature are even applicable to the post-H.R. 3 context.

The CBO’s elasticity estimate assumes that for a 1% decrease in revenue there will be a 0.53% decrease in the outputs of innovation (in this case NMEs).  Among the experts in our study, the median was 0.75 with a range of 0.4-4, reflecting the significant variance in estimates. The inability of the experts to agree about the elasticity of innovation and the usefulness of historic studies of the elasticity is critical because it directly impacts the reliability of the CBO estimates. If, for example, the true elasticity of innovation is 2, then instead of predicting 30 fewer drugs during 2030-39, the CBO would have predicted 114.

The uncertainty and therefore variability of the experts’ estimates were even greater when asked to consider the impact at a disaggregated level for different market sizes, different company sizes, or whether innovation was more responsive to US or global revenues.  They generally argued that the available data and experience were insufficient or too old to support assumptions about these disaggregated impacts.

There is Significant Uncertainty in the Dynamics of Innovation

The CBO’s findings are sensitive to assumptions on what happens in markets outside of the US, which is of course uncertain. The CBO assume that there will be a response to H.R. 3 outside the US including increased prices in the reference countries which would partially offset some of the lost US revenue, and they calculated a 19% reduction in the net present value in the global revenues for new drugs despite assuming a 68% reduction in the average Medicare cost of drugs targeted by H.R.3.

There was no agreement among the experts on whether this assumption was reasonable. Some experts believed that the impact on worldwide revenue would be substantially higher than the 19% used by the CBO with rationales for higher estimates related to conservative predictions of the ex-US dynamics. For example, the ability of companies to delay or stop launches in reference countries, or to raise list (but not net) prices in these countries.

The response to H.R. 3 will also change over time, and therefore modelling 20 years into the future is problematic. Some experts emphasized that these the long-term estimates are difficult to make due to potential future policy changes and uncertainty in how and when the market will adapt to these changes over time. Given the sensitivity of conclusions to policy and market changes over time, the CBO’s findings clearly are overly simplistic and specific. This is dangerous because it gives a false sense of certainty about the potential impacts of H.R. 3 which would indeed be many times greater than their analysis suggests.

The CBO Model is Not Fit for Purpose

Our expert respondents recognised that the CBO had attempted a highly complex task in predicting the long-term impact of a policy like H.R. 3. But the analysis is hampered by substantial uncertainty surrounding estimates of the key parameters, especially the relationship between revenue reduction and innovation lost (the elasticity of innovation). Experts pointed out how the uncertainty around both the response of innovation to expected revenue and the impact of the policy on expected revenue, compounded uncertainty which is very difficult to gauge and extrapolate with confidence. 

Given these uncertainties, a few experts felt that the CBO had done a respectable job, while other experts felt the CBO analysis was “fatally flawed” and that the choice of some variables did not follow the data.

We have discussed each of the assumptions in isolation but changing all of the CBO’s parameters simultaneously would have a multiplicative effect. For example, using the median estimates from the experts would generate a prediction of 60 fewer drugs (2030-39) — twice the central CBO estimate — with a median uncertainty range of 16 (lower bound) to 270 (upper bound).

Whatever position one might take on the quality and reliability of the CBO analysis, one thing is indisputable: the CBO analysis has not provided any insight on what types of innovation would be sacrificed. As we discussed in our previous blog, the analysis does not assess what that innovation loss means in terms of health gain foregone by patients in the US and around the world. Despite best efforts, the CBO’s estimates are unreliable and not fit for the purposes of guiding policymaking

To Follow in the OHE H.R 3 Series

In the final blog in our Series we turn our attention to the newest analysis by the CBO which attempts to similuate the decison making process of investors.