By ecancer reporter Will Davies

In the realm of modern medicine, drug approvals come thick and fast. Last year saw a 21-year high in FDA approvals, including the first approvals of gene therapies such as CAR-T cell therapy. Alongside these approvals have come concerns over drug pricing, with defenders citing the cost of research and validation.

However, some drugs enter the market before that research is complete.

Durvalumab and enfortumab vedotin are two of the latest oncology drugs to receive FDA approval based on their interim trial results. For durvalumab, this approval was based on improved PFS of stage III lung cancer patients, with results published in the New England Journal of Medicine, and approval of enfortumab vedotin for pretreated metastatic urothelial cancer was based on a 41% objective response rate among 71 patients. Both drugs were also granted breakthrough designations alongside their approval.

While these approvals made waves as the latest in a chain of lung-targeting immunotherapies and a new entry in the young field of antibody-drug conjugates, other drugs make waves for the wrong reasons.

“The reason we got started was we were called by a journalist asking about everolimus, which increased PFS in interim results, published in New England Journal of medicine. Then it turned out in the final publication that the drug didn’t have an effect on OS, which was published in a much more obscure journal. If you went to the NEJM article, you’d never know this. It turns out the effect of the drug on PFS was the same, but the lack of effect on OS was important, because the drug has a fair number of toxicities which are pretty unpleasant” says Lisa Schwartz, MD, professor at The Dartmouth Institute and co-author of a study in JAMA which dug into the rates of trials results changing, or disappearing, come publication day. “It got us interested in how often these change, and if they do change, are they linked so you would know something had happened?”

That gap between interim approval and final results is wider than you’d hope.

A meta-analysis of 1267 publications within Pubmed, Clinicaltrials.gov and Web of Science looked for the phrases “interim”, “not mature” or “immature” in the title or abstract. Interim results were reported in 613 and of those 613, 446 were stopped early for reasons of benefit, harm, futility or other problems.

“Within interim results, there are different flavours. The most common are where trials are stopped early. What we were interested in were trials that were ongoing, but were having interim publications that had either efficacy or safety results.” Dr Schwartz added.

“Of the remaining eligible trials, 59% did not have a specified interim analysis in the trial protocol, but issued an interim report all the same. For those that could be linked to a final publication of results, most were published in a journal of similar prominence. This was often not the case, though.”

“For many of them, we couldn’t find a subsequent publication. In order to see whether the results change, we need to see the final report. Of the trials we thought had had enough time where there could be a final report published, at least a year past the study completion date on the trial registry, only 61% of those had a final publication.”

“If you don’t have the final publication, that’s a real problem too. We tried to email authors when we couldn’t find a final publication, and didn’t have a great response rate. It certainly could be that reports were delayed, and could still come out, but still I think there probably is some substantial number where we’ll never know the true results.”

Ultimately, from a starting pool of 613 interim reports, only 73 could be linked to final results. Of those, 58 (79%) abstract conclusions did not change, 15 (21%) did.

“To us as clinicians, 20% changing seemed important, and you have a good chance of coming to the wrong conclusion. Certainly in the case where we thought something was beneficial and then it turned out to be harmful that certainly seems bad. You can imagine the initial publication would have encouraged people to use that drug.”

Among those changes, 11 were downgrades from beneficial to no difference, no difference to harmful, and, in one concerning case of high-dose imatinib, from beneficial to harmful.

“The primary outcome in the case of imatinib was not the thing you’d care about the most. That result was very similar. EFS was a secondary outcome, and while high dose imatinib had improved the surrogate outcome, high dose has reduced event-free survival. But you couldn’t have know that because you didn’t have enough events at the interim report to be able to estimate that.”

This has led to Schwartz and Woloshin producing an alternative drug information sheet for imatinib, published on The Drug Facts Box website,  written in plain language, to reduce the risk of confusion and misinformation for patients. However, patients are not the only ones sometimes in need of clear language.

Schwartz and Woloshin have previously compared the reporting of results from new drug trials on the separate ClinicalTrials.gov database and those at Drugs@FDA, finding the former of less use in validating secondary outcomes and adverse events, such as the case from high dose imatinib above.

They have also analysed the subsequent approvals and designations of drugs based on early indications, especially those that earn FDA Breakthrough designations.

“That language to us is amazing. It feels like marketing language, and that designation is not given at the time when we know the outcomes. It’s given really in the process when there’s a possibility of being a breakthrough.”

Through a survey of doctors across the US, they investigated perceptions of what a breakthrough designation meant for drug effectiveness and evidence of benefit.

Across the country, 77% of those surveyed answered Yes to “When the FDA calls a drug a breakthrough, does that mean that there is high-quality evidence that the drug is more effective than currently approved treatments?”

“They thought it meant there was a real evidence of effectiveness, not that it looked like it could be better based on preliminary results or in the lab, which is what the designation is often based on.”

“We also gave doctors the results of two identical uncontrolled studies on surrogate outcomes, and labelled one as a breakthrough, and everyone chose the breakthrough with the same, weak data. The language is so compelling, how could you not think a breakthrough is a major advance?”

So, between the trials which are never finally reported, the trials which change their outcomes and the designations that may change your choices, how does Lisa Schwartz recommend clinicians remain vigilant?

“The first thing is to know if interim analysis was protocol specified. If it was, it should say that a certain number events will have had to have happened to give it some statistical stability. When it’s not specified, or there are few events, those are the cases to be suspicious”.