How Does ICER’s New Cost-Effectiveness Framework Stack Up to Industry Practice?

By Matthew Sussman, MA

In early February, the Institute for Clinical and EcoHeadshot of Matt Sussmannomic Review (ICER) presented proposed updates to its Value Assessment Framework, based largely on stakeholder feedback received during a public comment period in late 2016. These updates focused on four domains of the framework: (1) comparative clinical effectiveness; (2) incremental cost-effectiveness (CE); (3) other benefits or disadvantages; and (4) contextual considerations. While each domain is important, the incremental CE analysis has proved the most controversial. Since the current comment period is almost up, here’s my review of the updates to the CE framework.

How Does Each Proposed Change Stack Up?

Below is a table I’ve put together to summarize ICER’s proposed updates and to indicate my opinion for the final framework release. The updates that don’t need additional revision are labeled with a check (“”), changes that require additional clarification are marked with a question (“?”), and changes that need major revisions are labeled with an “X”.

ICER Value Framework Updates: No Revisions Needed

Three of ICER’s proposed updates adhere to industry “best” practices today. These include:

  • 3.1 Maintain the primary effectiveness measure
    • The primary effectiveness measure will remain quality-adjusted life years (QALYs), leading to an evaluation of incremental costs per QALY gained.
  • 3.5 Include drug prices net of discounts, rebates, and other price concessions
    • ICER proposes to use drug prices net of discounts, rebates, and other price concessions instead of wholesale acquisition costs in its cost-effectiveness and budget impact analyses.
  • 3.7 Do not project price changes due to patent and exclusivity time horizons
    • ICER proposes to no longer include projections for price changes due to patent and exclusivity time horizons in their CE analyses. Scenario analyses may be conducted if a major change to pricing is anticipated within 12 to 24 months.

As noted in Section 3.1, the QALY is an industry-approved measure for capturing both the quantity and quality of life years generated by healthcare interventions1, and therefore is a natural output for this framework. In Section 3.5, using undiscounted wholesale acquisition costs would overestimate real-world acquisition costs, so applying drug prices net of discounts is a worthwhile change. Likewise, there is tremendous long-term uncertainty regarding market and pricing dynamics, so limiting price changes due to patent and exclusivity time horizons is a reasonable approach in Section 3.7.

ICER Value Framework Updates: Additional Clarification Required

There are also three updates to the new framework that need additional consideration and clarification. These include:

  • 3.2 Use a broader range of cost-effectiveness thresholds
    • For several years, ICER used a range of $100,000 to $150,000 per QALY gained. However, further review of the literature coupled with discussions with stakeholders have led ICER to propose a broader range of incremental CE thresholds between $50,000 and $150,000 per QALY gained.
  • 3.3 Include other effectiveness measures
    • As stated in Section 3.1, ICER will evaluate QALYs as the primary effectiveness measure. In addition to QALYs, ICER proposes to base the denominator of the incremental CE ratio on secondary effectiveness measures such as life years and other clinically-relevant consequences (e.g., strokes averted within atrial fibrillation population).
  • 3.6 Conduct scenario analyses from a societal perspective
    • ICER proposes to maintain the health system perspective as the base case, and perform scenario analyses from a societal perspective focusing on work productivity.

Evaluating an economically-justifiable price based on an incremental CE threshold is a decades-old practice for determining society’s willingness to pay for a new intervention. However, Section 3.2 raises the question as to whether ICER’s lower bound of $50,000 per QALY gained is too low. Based on alternative assumptions and calculations, some experts have indicated this to be the case, and instead have proposed using either $100,000 or $150,000 per QALY gained if a single threshold is desired.2

We mostly agree with the inclusion of effectiveness measures complementary to QALYs in Section 3.3, but caution should be used by ICER. Benchmarks in the literature should be a requirement for comparison to ICER’s incremental cost per consequence output. Without proper and relevant benchmarks, the model output will have no direct comparison, thereby limiting its interpretation.

Similarly, we agree that scenario analyses representing the societal perspective should be assessed in Section 3.6. Yet, ICER modeling teams should carefully consider which populations are most impacted by productivity losses. For instance, in their most recent documentation for the osteoporosis topic, the ICER modeling team indicated that productivity losses would be assessed – in this case, among a starting population of 70-year old postmenopausal women – which is counterintuitive among an elderly population.

ICER Value Framework Updates: Revisions Needed

Two of ICER’s proposed updates raised significant questions and will hopefully be revised and clarified in the final framework. These updates include:

  • 3.4 Conduct scenario analyses to assess the impact of lower health utilities
    • ICER proposes to conduct scenario analyses that compare the impact of lower baseline health utilities among patients with chronic and severe conditions to higher baseline health utilities for the general population. The incremental costs per QALY gained will be compared and the relative difference will be evaluated. If the impact of lower utilities on the incremental CE ratio for the chronic and severe population is substantially different, ICER along with key stakeholders will decide which scenario analysis will serve as the base case.
  • 4 Link ‘other benefits or disadvantages and contextual considerations’ to long-term value for money
    • ICER’s proposed approach for estimating the CE threshold attempts to link ‘other benefits or disadvantages and contextual considerations’ – such as unmeasured patient health benefits, relative complexity of the treatment regimen, impact on productivity, among others – to long-term value for money. The approach contains a modified form of multi-criteria decision analysis (MCDA), in which 10 total elements are considered but not quantitatively weighted to yield an average ranking on a scale from 1-5. The average score will, in turn, be used to assign a single incremental CE ratio benchmark from $50,000-$150,000 per QALY.

ICER Value Framework Section 3.4 Concerns:

The rationale behind this update was not clearly stated and raised a few questions that will need to be addressed in the final framework:

  • What is the rationale for this proposed change? Is it that the patient mix will include patients who are newly diagnosed and thus have baseline utilities closer to the general population?
  • What criteria will be used to decide which scenario analysis serves as the base case?
  • Will literature-based benchmarks be factored into the decision?

This proposed change also raises the issue of sub-group analyses, specifically among at-risk groups (e.g., those with greater disease severity and/or varying patient profiles). Instead of deciding on one scenario analysis that serves as the base case, perhaps sub-group analyses should be conducted in which both scenarios are presented.

ICER Value Framework Section 4 Concerns:

As with Section 3.4, I have a few reservations about the new CE threshold estimate proposed in Section 4. ICER indicated that a single incremental CE ratio will be used to determine a value-based price, as well as an overall value rating, for each intervention under study. This means that ICER’s findings could place additional downward pressure on drug prices. The potential impact of this proposed change on pharma stakeholders cannot be overstated or de-emphasized: your products will be judged against the CE ratio benchmark. For a more detailed review of the new CE Threshold, I recommend a quick read of my previous post, ICER’s New Cost-Effectiveness Threshold Raises 5 Major Questions.

Next Steps For ICER’s Value Assessment Framework

ICER has attempted to address hundreds of pages of comments and suggestions in their Value Assessment Framework, and should be lauded for their efforts. However, several proposed revisions to their incremental CE framework require greater clarification and/or adjustment prior to ratification and implementation. We have submitted our concerns, and look forward to seeing them addressed in the revised framework due out April 15th.

1Weinstein MC, Torrance G, McGuire A. QALYs: The basics. Value Health. 2009 Mar;12 Suppl 1:S5-9.

2Neumann PJ, Cohen JT, Weinstein MC. Updating cost-effectiveness – The curious resilience of the $50,000-per-QALY threshold. N Engl J Med. 2014;371(9):796-797.

ICER’s New Cost-Effectiveness Threshold Raises 5 Major Questions

By Matt Susssman, MA

Headshot of Matt SussmanEarly last month the Institute for Clinical and Economic Review (ICER) released a revised Value Assessment Framework for public comment based on feedback received from patients, clinicians, life science companies, and other stakeholders. While a majority of the proposed changes lacked real luster, there is one proposed change that pharma should take note of: the new method for estimating the cost-effectiveness (CE) threshold.

Proposed Method for Estimating Cost-Effectiveness (CE) Threshold

ICER’s proposed approach for estimating the CE threshold attempts to link ‘other benefits or disadvantages and contextual considerations’ – such as unmeasured patient health benefits, relative complexity of the treatment regimen, impact on productivity, among others – to long-term value for money. The approach contains a modified form of multi-criteria decision analysis (MCDA), in which 10 total elements are considered but not quantitatively weighted to yield an average ranking on a scale from 1-5. The average score will, in turn, be used to assign a single incremental CE ratio benchmark from $50,000-$150,000 per quality-adjusted life year (QALY). In particular, ICER proposes the following steps:

Proposed ICER Steps for Cost-Effectiveness Threshold Estimate

Why Should Pharma Care About ICER’s Revised CE Threshold?

The single incremental CE ratio will be used to determine a value-based price, as well as an overall value rating, for each intervention under study. This means that ICER’s findings could place additional downward pressure on drug prices. The potential impact of this proposed change on pharma stakeholders cannot be overstated or de-emphasized: your products will be judged against the CE ratio benchmark.

5 Questions For ICER’s Value Assessment Cost-Effectiveness Threshold

During the ongoing public commentary period, pharma should examine every nook and cranny of ICER’s Value Assessment Framework, especially the new CE threshold estimate, and ask for greater clarity where necessary. As a start, here are our five main questions regarding the new threshold approach:

Question 1: Who are the members of the independent public appraisal committee?

Drug manufacturers will need to understand the backgrounds of those directly impacting the CE ratio benchmark, since their initial decisions may ultimately place pressure on drug prices. In particular, pharma should seek to find out:

  • What are the profiles of the committee members?
  • Will the profiles of each member be made publicly available?
  • Is the committee comprised of clinicians, economists, patient advocacy leaders, and/or payers?
  • How many committee members are there in total, and how many members are from each stakeholder group?

Question 2: Will the case studies be specific to a therapeutic/disease area, or will they be generic across therapeutic/disease areas?

If the goal of the case studies is to inform scoring for each of the 10 elements on a visual analog scale, the case studies will need to be relevant to the therapeutic/disease area under consideration. Otherwise, generic case studies may lead the committee to score the element in an unintended way.

Question 3: Is it appropriate to link selected ‘other considerations’ focused on societal impacts to direct cost per QALY thresholds?

Selected elements, such as ‘impact on productivity’, ‘impact on caregiver burden’, and ‘impact on public health’, are societal constructs. Yet these societal elements will be used, along with other considerations, to determine an overall ranking leading to a direct cost per QALY threshold. If these societal elements are to be used, perhaps they should lead to a direct plus indirect cost per QALY threshold.

Question 4: Are the overall ranking scores of 1-5 referenced and compared across therapeutic/disease areas or similar treatments?

Embedded in their proposed changes, ICER clearly states that the overall ranking score determined by the public appraisal committee “reflects one group’s judgement”, hinting that a second committee may produce a different score. This raises the question as to how ICER could explain differences in scores on the same topic from two hypothetical committees. Perhaps a committee’s score should be compared to past scores in similar therapeutic/disease or product areas, as a means of checks and balances.

Question 5: Is it appropriate to assign a ‘one-size fits all’ threshold range across diseases and treatments?

It is reasonable to expect, as discussed by Drummond and Sorenson1, that the CE threshold may differ depending on the treatments being evaluated or the patient populations being studied. Consider the case of rare diseases, for which orphan or ultra-orphan drugs may not be cost effective and may exceed ICER’s CE range of $50,000-$150,000 per QALY. Payers may still approve coverage of these therapies, irrespective of a high CE ratio, since they are used to treat life-threatening diseases for which other therapeutic options may not be available. Should these drugs, and others used in special circumstances, be held to a different CE range?

ICER’s Value Assessment Framework Comment Period Is Now Open

Although there are many more questions that can and should be asked of ICER regarding proposed changes to its Value Assessment Framework, the cost-effectiveness threshold estimate is the most impactful and controversial and should be given a full review. Take the time and make comments. We’re submitting these questions, but ICER needs more feedback. Once this change and all others have been ratified, ICER will not revisit their methodology until April 2019. Now is your opportunity to inspect, comment, and critique the new CE benchmarking approach before the public commentary period ends on April 3rd, at 5pm ET.

1Drummond M, Sorenson C. Nasty or NICE? A perspective on the use of health technology assessment in the United Kingdom. Value in Health. 2009; 12(2):S8-S13.