Robert Califf (Michael Brochstein/Sipa USA via AP Images)

Re­al world ev­i­dence: Lessons learned from an FDA pi­lot show the lim­its of em­u­lat­ing RCTs

On­ly about half of a se­lect group of clin­i­cal tri­als could be well-em­u­lat­ed with avail­able re­al world ev­i­dence, ac­cord­ing to the new­ly dis­cussed re­sults of an FDA pi­lot pro­gram.

The FDA-fund­ed pro­gram, known as RCT-DU­PLI­CATE, helped re­searchers from the Brigham and Women’s Hos­pi­tal in Boston try to eval­u­ate whether ran­dom­ized con­trolled tri­als can be du­pli­cat­ed with RWE across a range of ther­a­peu­tic ar­eas.

The ini­tia­tive is part of the FDA’s work, man­dat­ed by Con­gress in the 21st Cen­tu­ry Cures Act, on how it plans to eval­u­ate the use of RWE to as­sess the ef­fec­tive­ness of med­ical prod­ucts.

John Con­ca­to, as­so­ciate di­rec­tor for RWE an­a­lyt­ics in the FDA’s Of­fice of Med­ical Pol­i­cy, made clear at a Duke-Mar­go­lis event on the top­ic last week that the stan­dard for sub­stan­tial ev­i­dence re­mains un­changed. He al­so not­ed that seek­ing to em­u­late a hy­po­thet­i­cal RCT when de­sign­ing a non-in­ter­ven­tion­al study “is a fun­da­men­tal­ly dif­fer­ent task” than what the re­searchers did with RCT-DU­PLI­CATE.

In the case of RCT-DU­PLI­CATE, Shirley Wang, as­so­ciate pro­fes­sor of med­i­cine at Har­vard Med­ical School, ex­plained how the pro­gram sought to em­u­late 30 RCTs us­ing clin­i­cal prac­tice da­ta.

The re­searchers found that 50% of the se­lect­ed RCTs could be em­u­lat­ed close­ly re­gard­ing de­sign and analy­sis, and they saw com­pa­ra­ble treat­ment ef­fects. But RCT and RWE find­ings were more like­ly to di­verge when there were sub­stan­tive em­u­la­tion chal­lenges, which could be be­cause the data­base and RCTs are tar­get­ing dif­fer­ent ques­tions, or due to bias, or both.

Sub­stan­tive chal­lenges were seen in the re­searchers’ at­tempts to repli­cate stud­ies in os­teo­poro­sis, chron­ic kid­ney dis­ease, heart fail­ure, asth­ma and COPD, Wang said.

How­ev­er, oth­er tri­als that could be close­ly em­u­lat­ed dealt with atri­al fib­ril­la­tion, VTE and hy­per­ten­sion, where­as tri­als re­lat­ed to di­a­betes (8 tri­als) and an­tiplatelet ther­a­py (3 tri­als) saw vary­ing de­grees of chal­lenges.

“Data­base stud­ies can come to sim­i­lar con­clu­sions as RCTs when we are able to em­u­late them well. But it’s more chal­leng­ing to em­u­late tri­als that are de­signed with many con­straints to show ef­fects un­der dif­fer­ent con­di­tions,” Wang said, not­ing that it’s eas­i­er to em­u­late tri­als with more prag­mat­ic de­sign fea­tures.

“With da­ta that are fit-for-pur­pose and prop­er de­sign and analy­sis, non-ran­dom­ized RWE stud­ies can come to sim­i­lar con­clu­sions about a drug’s treat­ment ef­fect as ran­dom­ized tri­als,” Wang added.

Se­bas­t­ian Schneeweiss, pro­fes­sor of med­i­cine at Har­vard who al­so dis­cussed re­sults of the study last week, not­ed, “Most im­por­tant­ly, re­al-world ev­i­dence com­ple­ments ev­i­dence from ran­dom­ized tri­als and does not re­place it. The con­clu­sion of this demon­stra­tion project should not be that every RWE study needs to em­u­late an RCT. This would de­feat the val­ue of RWE in com­ple­ment with RCTs.”

The FDA-fund­ed Brigham Women’s re­search is still work­ing on em­u­lat­ing 7 RCTs that are on­go­ing.

Over­all, the FDA’s drug cen­ter has sup­port­ed more than 20 RWE demon­stra­tion pro­jec­tions, in­clud­ing ones re­lat­ed to im­prov­ing the use or qual­i­ty of RWD, as­sess­ing RWE study de­signs (e.g. RCT-DU­PLI­CATE), or eval­u­at­ing tools for spe­cif­ic pur­pos­es.

Back in 2018, the FDA al­so un­veiled its RWE frame­work, which has since been sup­ple­ment­ed with 4 new draft guid­ance doc­u­ments on EHRs and med­ical claims da­ta, da­ta stan­dards for RWD sub­mis­sions, as­sess­ing reg­istries to sup­port reg­u­la­to­ry de­ci­sions mak­ing, and us­ing RWD and RWE to sup­port reg­u­la­to­ry de­ci­sion-mak­ing.

But in ex­plain­ing how there are still mul­ti­ple ques­tion marks around the use of RWE, FDA’s Con­ca­to al­so high­light­ed an NE­JM ar­ti­cle from 2020, in which re­searchers from the Uni­ver­si­ty of Ox­ford wrote, “The re­place­ment of ran­dom­ized tri­als with non­ran­dom­ized ob­ser­va­tion­al analy­ses is a false so­lu­tion to the se­ri­ous prob­lem of en­sur­ing that pa­tients re­ceive treat­ments that are both safe and ef­fec­tive.”

Even FDA Com­mis­sion­er Rob Califf, who wasn’t com­mis­sion­er at the time, was on board.

2023 Spot­light on the Fu­ture of Drug De­vel­op­ment for Small and Mid-Sized Biotechs

In the context of today’s global economic environment, there is an increasing need to work smarter, faster and leaner across all facets of the life sciences industry.  This is particularly true for small and mid-sized biotech companies, many of which are facing declining valuations and competing for increasingly limited funding to propel their science forward.  It is important to recognize that within this framework, many of these smaller companies already find themselves resource-challenged to design and manage clinical studies themselves because they don’t have large teams or in-house experts in navigating the various aspects of the drug development journey. This can be particularly challenging for the most complex and difficult to treat diseases where no previous pathway exists and patients are urgently awaiting breakthroughs.

Dipal Doshi, Entrada Therapeutics CEO

Ver­tex just found the next big ‘trans­for­ma­tive’ thing for the pipeline — at a biotech just down the street

Back in the summer of 2019, when I was covering Vertex’s executive chairman Jeff Leiden’s plans for the pipeline, I picked up on a distinct focus on myotonic dystrophy Type I, or DM1 — one of what Leiden called “two diseases (with DMD) we’re interested in and we continue to look for those assets.”

Today, Leiden’s successor at the helm of Vertex, CEO Reshma Kewalramani, is plunking down $250 million in cash to go the extra mile on DM1. The lion’s share of that is for the upfront, with a small reserve for equity in a deal that lines Vertex up with a neighbor in Seaport that has been rather quietly going at both of Vertex’s early disease targets with preclinical assets.

Endpoints Premium

Premium subscription required

Unlock this article along with other benefits by subscribing to one of our paid plans.

Ahead of ad­comm, FDA rais­es un­cer­tain­ties on ben­e­fit-risk pro­file of Cy­to­ki­net­ic­s' po­ten­tial heart drug

The FDA’s Cardiovascular and Renal Drugs Advisory Committee will meet next Tuesday to discuss whether Cytokinetics’ potential heart drug can safely reduce the risk of cardiovascular death and heart failure in patients with symptomatic chronic heart failure with reduced ejection fraction.

The drug, known as omecamtiv mecarbil and in development for more than 15 years, has seen mixed results, with a first Phase III readout from November 2020 hitting the primary endpoint of reducing the odds of hospitalization or other urgent care for heart failure by 8%. But it also missed a key secondary endpoint analysts had pegged as key to breaking into the market.

David Light, Valisure CEO

Val­isure in the hot seat: New Form 483 over a 2021 in­spec­tion as CEO fires back

The notorious drug testing company Valisure, which has made a name for itself by forcing FDA’s hand with some of its safety-related uncoverings, received a letter this week after the FDA uncovered violations at its Connecticut-based testing lab in 2021.

The letter, which was sent on Dec. 5, stated that the FDA is “concerned” that Valisure was not aware of  drug supply chain security requirements.

WIB22: Am­ber Salz­man had few op­tions when her son was di­ag­nosed with a rare ge­net­ic dis­ease. So she cre­at­ed a bet­ter one

This profile is part of Endpoints News’ 2022 special report about Women in Biopharma R&D. You can read the full report here.

Amber Salzman’s life changed on a cold, damp day in Paris over tiny plastic cups of lukewarm tea.

She was meeting with Patrick Aubourg, a French neurologist studying adrenoleukodystrophy, or ALD, a rare genetic condition that causes rapid neurological decline in young boys. It’s a sinister disease that often leads to disability or death within just a few years. Salzman’s nephew was diagnosed at just 6 or 7 years old, and died at the age of 12.

Endpoints News

Keep reading Endpoints with a free subscription

Unlock this story instantly and join 154,300+ biopharma pros reading Endpoints daily — and it's free.

FDA re­view­ers head back to White Oak in 2023, with lead­er­ship look­ing to ap­pease a new Con­gress

Republicans have taken a stand against the pandemic era habit of lax work-from-home schedules. Now that they’ve wrestled control of the House majority, the FDA’s leadership is playing ball, sending many of the agency’s more than 18,000 employees back to their desks early next year.

Whether this exodus back to White Oak in Silver Spring, MD (many staff will still be allowed to work from home for multiple days per week) will mean more defections to industry and elsewhere remains to be seen.

Bags of shred­ded docs: In­di­an drug­mak­er Lupin hand­ed a Form 483 by FDA in­spec­tors

The generics manufacturer Lupin has been given another Form 483 from the FDA this year.

US regulators inspected Lupin’s pharmaceutical manufacturing site in the town of Mandideep, India from Nov. 14 through Nov. 23, with the 14-page report marking 16 observations.

The inspection report stated that the site did not have the appropriate controls over its computer systems to ensure that changes in “master production” or records are only done by authorized personnel, along with written procedures not being established to conduct annual reviews of records associated with drug batches.

Bro­ken promis­es? FDA needs more pow­er to re­move drugs from mar­ket­place, JA­MA analy­sis finds

The FDA is struggling to remove drugs from the marketplace that don’t show effectiveness in late stage trials, new JAMA analyses found, thanks to the persistent tension between speed and confidence in early clinical data.

Congress, regulated industry and patients have urged the FDA to shorten the amount of time that the market has to wait for drugs to become available that may help severe and prevalent diseases – and the FDA has listened, offering up a quick accelerated approval pathway that’s frequently used by new cancer drugs.

Ab­b­Vie slapped with age dis­crim­i­na­tion law­suit, fol­low­ing oth­er phar­mas

Add AbbVie to the list of pharma companies currently facing age discrimination allegations.

Pennsylvania resident Thomas Hesch filed suit against AbbVie on Wednesday, accusing the company of passing him over for promotions in favor of younger candidates.

Despite 30 years of pharma experience, “Hesch has consistently seen younger, less qualified employees promoted over him,” the complaint states.