Lies, damn lies and sta­tis­tics: A Stan­ford wiz says P<0.05 of­fers de­cep­tive ev­i­dence of bio­phar­mas' drug claims

The bio­phar­ma R&D world re­volves around one sim­ple for­mu­la: A P val­ue of less than 0.05 in a piv­otal study. But a top pro­fes­sor of med­i­cine and sta­tis­tics at Stan­ford says it’s a poor mea­sure of val­ue, and he wants to scrap it for some­thing far more de­mand­ing — and far more valu­able.

Writ­ing in the Jour­nal of the Amer­i­can Med­ical As­so­ci­a­tion, John P A Ioan­ni­dis notes that the P val­ue cut­off “is wrong­ly equat­ed with a find­ing or an out­come…be­ing true, valid, and worth act­ing on. These mis­con­cep­tions af­fect re­searchers, jour­nals, read­ers, and users of re­search ar­ti­cles, and even me­dia and the pub­lic who con­sume sci­en­tif­ic in­for­ma­tion.”

And they’re of­ten sim­ply wrong.

Most claims sup­port­ed with P val­ues slight­ly be­low .05 are prob­a­bly false (ie, the claimed as­so­ci­a­tions and treat­ment ef­fects do not ex­ist). Even among those claims that are true, few are worth act­ing on in med­i­cine and health care.

There are just too many ways to game the clin­i­cal tri­al sys­tem, Ioan­ni­dis adds. By fo­cus­ing on small­er ben­e­fits and risks, he writes, you boost the risk that bi­as­es will have an af­fect.

“Mov­ing the P val­ue thresh­old from .05 to .005 will shift about one-third of the sta­tis­ti­cal­ly sig­nif­i­cant re­sults of past bio­med­ical lit­er­a­ture to the cat­e­go­ry of just ‘sug­ges­tive.’ This shift is es­sen­tial for those who be­lieve (per­haps crude­ly) in black and white, sig­nif­i­cant or non­signif­i­cant cat­e­go­riza­tions.”

Ioan­ni­dis, though, is quick to as­sert that there are no easy so­lu­tions to the P val­ue co­nun­drum. There are ad­van­tages, and some big dis­ad­van­tages, for do­ing away with the old stan­dard that can’t be ig­nored.

Adopt­ing low­er P val­ue thresh­olds may help pro­mote a re­formed re­search agen­da with few­er, larg­er, and more care­ful­ly con­ceived and de­signed stud­ies with suf­fi­cient pow­er to pass these more de­mand­ing thresh­olds. How­ev­er, col­lat­er­al harms may al­so emerge. Bias may es­ca­late rather than de­crease if re­searchers and oth­er in­ter­est­ed par­ties (eg, for-prof­it spon­sors) try to find ways to make the re­sults have low­er P val­ues. Se­lect­ed study end­points may be­come even less clin­i­cal­ly rel­e­vant be­cause it is eas­i­er to reach low­er P val­ues with weak sur­ro­gate end points than with hard clin­i­cal out­comes. More­over, re­sults that pass a low­er P val­ue thresh­old may be lim­it­ed by greater re­gres­sion to the mean and new dis­cov­er­ies may have even more ex­ag­ger­at­ed ef­fect sizes than be­fore.

My bet is that the in­dus­try has be­come so fo­cused on beat­ing 0.05, no one will want to drop it for an untest­ed ap­proach that could throw the whole $160 bil­lion drug de­vel­op­ment busi­ness in­to a tizzy. There are no sim­ple bound­ary lines be­tween good and bad. But it’s def­i­nite­ly worth keep­ing in mind the next time you see a bio­phar­ma com­pa­ny cel­e­brat­ing a P val­ue in the 0.04 range of things.


Im­age: John P. A. Ioan­ni­dis. Eras­mus MC via YOUTUBE

A New Fron­tier: The In­ner Ear

What happens when a successful biotech venture capitalist is unexpectedly diagnosed with a chronic, life-disrupting vertigo disorder? Innovation in neurotology.

That venture capitalist was Jay Lichter, Ph.D., and after learning there was no FDA-approved drug treatment for his condition, Ménière’s disease, he decided to create a company to bring drug development to neurotology. Otonomy was founded in 2008 and is dedicated to finding new drug treatments for the hugely underserved community living with balance and hearing disorders. Helping patients like Jay has been the driving force behind Otonomy, a company heading into a transformative 2020 with three clinical trial readouts: Phase 3 in Ménière’s disease, Phase 2 in tinnitus, and Phase 1/2 in hearing loss. These catalysts, together with others in the field, highlight the emerging opportunity in neurotology.
Otonomy is leading the way in neurotology
Neurotology, or the treatment of inner ear neurological disorders, is a large and untapped market for drug developers: one in eight individuals in the U.S. have moderate-to-severe hearing loss, tinnitus or vertigo disorders such as Ménière’s disease.1 With no FDA-approved drug treatments available for these conditions, the burden on patients—including social anxiety, lower quality of life, reduced work productivity, and higher rates of depression—can be significant.2, 3, 4

Joe Jimenez, Getty

Ex-No­var­tis CEO Joe Jimenez is tak­ing an­oth­er crack at open­ing a new chap­ter in his ca­reer — and that in­cludes a new board seat and a $250M start­up

Joe Jimenez is back.

The ex-CEO of Novartis has taken a board seat on Century Therapeutics, the Versant and Bayer-backed startup focused on coming up with a brand new twist on cell therapies for cancer — a field where Jimenez made his mark backing the first personalized CAR-T approved for use.

Endpoints News

Keep reading Endpoints with a free subscription

Unlock this story instantly and join 70,800+ biopharma pros reading Endpoints daily — and it's free.

Credit: Shutterstock

Can we make the an­tibi­ot­ic mar­ket great again?

The standard for-profit model in drug development is straightforward. Spend millions, even billions, to develop a medicine from scratch. The return on investment (and ideally a tidy profit) comes via volume and/or price, depending on the disease. But the string of big pharma exits and slew of biotech bankruptcies indicate that the model is sorely flawed when it comes to antibiotics.

The industry players contributing to the arsenal of antimicrobials are fast dwindling, and the pipeline for new antibiotics is embarrassingly sparse, the WHO has warned. Drugmakers are enticed by greener pastures, compared to the long, arduous and expensive path to antibiotic approval that offers little financial gain as treatments are typically priced cheaply, and often lose potency over time as microbes grow resistant to them.

Top Har­vard chemist caught up in FBI’s 'T­hou­sand Tal­ents' drag­net, ac­cused of ly­ing about Chi­nese con­nec­tions, pay

The FBI’s probe into the alleged theft of R&D secrets by Chinese authorities has drawn Harvard’s top chemist into its net.

The agency accused Charles M. Lieber, who chairs the university’s chemistry and chemical biology department, with lying about his involvement in China’s Thousand Talents campaign, which was established as a way of drawing in innovators from around the world. And the scientist, 60, was charged with making false statements about his ties to China.

Endpoints News

Keep reading Endpoints with a free subscription

Unlock this story instantly and join 70,800+ biopharma pros reading Endpoints daily — and it's free.

Eye­ing a trio of tri­al ini­ti­a­tions, Jim Wilson's gene ther­a­py start­up woos Bruce Gold­smith from Deer­field as CEO

Passage Bio — Jim Wilson’s self-described “legacy company” — has wooed a seasoned biotech executive to steer the clinical entry of its first three gene therapy programs.

Bruce Goldsmith jumps to the helm of Passage after a brief CEO stint at Civetta, a cancer-focused startup he helped launch while a venture partner at Deerfield. He takes over from OrbiMed partner and interim chief Stephen Squinto, who will now lead the R&D team.

The FTC and New York state ac­cuse Mar­tin Shkre­li of run­ning a drug mo­nop­oly. They plan to squash it — and per­ma­nent­ly ex­ile him

Pharma bro Martin Shkreli was jailed, publicly pilloried and forced to confront some lawmakers in Washington riled by his move to take an old generic and move the price from $17.50 per pill to $750. But through 4 years of controversy and public revulsion, his company never backed away from the price — left uncontrolled by a laissez faire federal policy on a drug’s cost.

Now the FTC and the state of New York plan to pry his fingers off the drug once and for all and open it up to some cheap competition. And their lawsuit is asking that Shkreli — with several years left on his prison sentence — be banned permanently from the pharma industry.

UP­DAT­ED: Ac­celeron res­ur­rects block­buster hopes for so­tater­cept with pos­i­tive PhII — and shares rock­et up

Acceleron $XLRN says that its first major trial readout of 2020 is a success.

In a Phase II study of 106 patients with pulmonary arterial hypertension (PAH), Acceleron’s experimental drug sotatercept hit its primary endpoint: a significant reduction in pulmonary vascular resistance. The drug also met three different secondary endpoints, including the 6-minute walking test.

“We’re thrilled to report such positive topline results from the PULSAR trial,” Acceleron CEO Habib Dable said in a statement. The company said in a conference call they plan on discussing a Phase III trial design with regulators.

Endpoints News

Keep reading Endpoints with a free subscription

Unlock this story instantly and join 70,800+ biopharma pros reading Endpoints daily — and it's free.

Short at­tack­er Sahm Ad­ran­gi draws crosshairs over a fa­vorite of Sanofi’s new CEO — with PhII da­ta loom­ing

Sahm Adrang Kerrisdale

Kerrisdale chief Sahm Adrangi took a lengthy break from his series of biotech short attacks after his chief analyst in the field pulled up stakes and went solo. But he’s making a return to drug development this morning, drawing crosshairs over a company that’s one of new Sanofi CEO Paul Hudson’s favorite collaborators.

Endpoints News

Keep reading Endpoints with a free subscription

Unlock this story instantly and join 70,800+ biopharma pros reading Endpoints daily — and it's free.

Amber Saltzman (Ohana)

Flag­ship's first ven­ture of 2020 is out, and it's all about sperm

A couple years ago, Amber Salzman got a call as she was returning East full-time after a two-year stint running a gene therapy company in California.

It was from someone at Flagship Pioneering, the deep-pocketed biotech venture firm. They had a new company with a new way of thinking about sperm. It had been incubating for over a year, and now they wanted her to run it.

“It exactly fit,” Salzman told Endpoints News. “I just thought I had to do something.”