« Guide: Promoting Evidence-Based Insights | Home


Using the Evidence Graph to capture argumentation about health decisions.

Decision frameworks are tricky, and healthcare value assessment frameworks are no exception. While they can provide objectivity and support decision quality, they can also smother innovation and prevent some patients from receiving needed care. US pharma industry groups have reacted strongly to the framework used by ICER, the Institute for Clinical and Economic Review. This post recaps their argument and shows how to capture it using PepperSlice.

These screen grabs show how to visualize an argument as an Evidence Graph. Think of each node pair as a relationship. Example: ICER framework lacks Transparency.

Click any line on the graph to list supporting evidence for that relationship. All PepperSlice content is available as a searchable, reusable inventory of evidence-based insights.

Evidence Graph in PepperSlice

Evidence Graph with Details


PepperSlice content is also viewable as bite-size, top-line written summaries (Slices), as shown here. Each Slice corresponds one-to-one with a relationship on the Evidence Graph. Grouping Slices together captures an entire argument, such as this one about potential weaknesses in the ICER decision framework.




For your reference, here's a conventional, long-form review of the pharma industry criticisms of ICER's framework.

ICER triggers response. For more than a year, ICER has used a new value assessment framework to guide its evidence reports on new drugs and other interventions. Major industry groups have challenged the readiness of that framework, questioning its appropriateness in several areas. Now ICER has requested stakeholder comments in preparation for a 2017 revision.

NPC (the National Pharmaceutical Council), PhRMA (biopharma trade group), and AMCP (the Academy of Managed Care Pharmacy) have significant concerns about ICER’s methodology. In particular, they take issue on:

Real-world evidence. NPC says "ICER [should] have a clear process for managing the evolution of evidence, especially in the case of emerging therapies.... [T]hese reviews will continue to be relied upon by other stakeholders even after additional data (e.g., real-world evidence) emerge." And AMCP suggests real-world evidence and patient-reported outcomes should be "re-examined to further enhance the utility and relevance of the value assessment framework."

Budget impact. ICER should not confuse budget impact with value. "Budget impact assessments — which are measures of resource use, not of value — should remain completely separate from value assessments," says NPC. And this from PhRMA: ICER should suspend "the use of budget impact estimates until more sound methods are developed and validated."

Economic model transparency. The information provided is "not sufficient to enable reviewers to reproduce the results and provide meaningful, real-time input. Full transparency — down to the equation level — is needed to enable reproducible results and support fully informed stakeholder collaboration." NPC asks that ICER release the model to all stakeholders.

QALY. PhRMA asks for "Adjustment of the cost-effectiveness component of the framework to reflect the inherent and widely recognized limitations in traditional quality adjusted life years-based cost-effectiveness analysis (CEA), including capturing a wider range of benefits in CEA and presenting a range of care value estimates based on sound assumptions and varied approaches."

Other highlights:

NPC: To guide future development, NPC published a set of Guiding Practices for Patient-Centered Value Assessment. Dan Leonard recently recapped NPC's viewpoint on how frameworks should be developed.

PhRMA: Four specific recommendations are offered, intended to move ICER in a more "methodologically rigorous, patient-centered direction". They request significantly more transparency into how it works with stakeholders. And they offer specific advice on How to Get Value Assessment Frameworks Right.

AMCP: The pharmacists' group expresses concern that the current framework "lacks a process for incorporating real-world evidence (RWE) and patient reported outcomes (PROs) into the catalog of evidence that informs the underlying economic models. [Doing so would] better represent the patient experience."

Version 2.0. In October, ICER convened a broad group of stakeholders to inform its planned update. Invitees included people from pharma, academia, payers, patient advocates, and trade groups. A revised framework will be posted for additional comments next month; ICER's 2.0 version will likely become final in early 2017.

What's next for value frameworks? ICER is only one of several frameworks gaining traction in healthcare. To help establish best practices going forward, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) has launched an initiative on US value assessment frameworks; more than 250 people attended the kickoff stakeholder meeting. A task force is preparing a policy white paper on the appropriate definition and use of value assessment frameworks, expected Q1 2017. ISPOR is the sole funder of the effort.


Subscribe by email