How Novartis assesses a drug's probability of success prior to the pivotal trial
Transitioning a developing compound from early phase clinical trials to a pivotal trial(s) can be a make-or-break decision for a small- or medium-sized pharma. And sometimes companies aren’t very good at making the right decision.
While many companies often rely on outdated ways to assess the probability of any given compound’s success in making that transition, and clearly some are better than others — an accompanying editorial explains how Pfizer’s “end to end” success is “almost twofold higher than the industry benchmark” — Novartis scientists recently published in the May issue of Clinical Pharmacology & Therapeutics its framework for assessing the probability of success at the end of a smaller Phase II trial, or prior to that pivotal trial.
“The probability of success (PoS) of a program, a single number expressed as a percentage reflecting the multitude of risks that may influence the final program outcome, is a key decision-making tool,” Novartis’ Lisa Hampson and colleagues wrote. “Despite its importance, companies often rely on crude industry benchmarks that may be ‘adjusted’ by experts based on undocumented criteria and which are typically misaligned with the definition of success used to drive commercial forecasts, leading to overly optimistic expected net present value calculations.”
Instead, Hampson offers an approach organized in four steps that uses “an innovative Bayesian approach to synthesize all relevant evidence,” including:
- Obtain tailored industry benchmarks that are updated every 6-12 months, which is particularly important for cell and gene therapies, the authors note, on the probability of efficacy success for a Phase IIb program and a Phase III, the probability of approval for an NDA submission, and the probability of no so-called “Safety Showstopper Event” in Phase III.
- Estimate the probability of overall success in Phase III based on industry benchmarks and Phase IIb data.
- Estimate the probability of regulatory approval and meeting remaining total product profile (TPP) thresholds (i.e. outcomes and thresholds necessary for market access) with a scorecard on assessing sponsor alignment with the regulator, unaccounted safety risks, and quality and compliance risks, among others.
- Incorporate and adjustment factor to reflect exceptional unaccounted events (i.e. unknown unknowns).
The paper also walks through a hypothetical example (an interleukin receptor antagonist already approved for asthma that’s under development for COPD) for the Novartis step-by-step process, finding that while the probability of a Phase III ending up with statistically significant outcome was estimated at 58%, actually hitting that mark and meeting the TPP targets without an SSE was estimated at 25%. With the scorecard factored in, the final PoS for the hypothetical drug was estimated at 24%.
“In this example, the PoS was quite low and notably lower than the probability of achieving statistical significance in phase III, even though experts believed the drug to be efficacious. This was due to ambitious thresholds set in the TPP for the key efficacy outcomes,” the authors noted.
The paper then walks through a real-world example, using Amgen and AstraZeneca’s tezepelumab for asthma. While the sponsors initiated a Phase III study for the drug in the real world (and it was approved in this indication late last year), the assessment in this paper was done with only public information and carried out prior to the Phase III.
The Novartis model found that tezepelumab’s Phase III success with statistically significant results was 99%, although factoring in all the steps brought the final PoS to 82%.
“An important consideration when implementing the new PoS framework in a complex organization is change management,” the authors added. And while some project teams at Novartis raised concerns about whether certain programs might be automatically terminated based on a PoS threshold, the authors noted that “even high-risk programs may be supported, if the medical need is high or the market opportunity is large.”
In the accompanying commentary, authors from the University of Florida and the University of California San Francisco underscore the importance of making an informed and objective decision before proceeding to later-stage trials, particularly as about 90% of all drug candidates fail during clinical development.
