Food and Drug Administration Commissioner Stephen Hahn drew a hailstorm of criticism from scientists this week for mischaracterizing a study's findings in a way that hyped the benefits of convalescent plasma. He later apologized, but critics say the damage was done.

Food and Drug Administration Commissioner Stephen Hahn drew a hailstorm of criticism from scientists this week for mischaracterizing a study's findings in a way that hyped the benefits of convalescent plasma. He later apologized, but critics say the damage was done. / AP

Updated 4:35 p.m. ET

The Food and Drug Administration's chief has undercut the agency's assertion that it is basing its decisions on science, not politics.

At a White House event Sunday with President Trump, FDA Commissioner Stephen Hahn used a deeply misleading statistic to claim that a treatment the agency had just authorized for treating the coronavirus would save 35 lives out of every 100 people who get the treatment.

That false claim has brought withering criticism from scientists, in news articles and on Twitter, who argue that it was a gross exaggeration of the benefits.

"I can't remember a mistake by FDA or the commissioner as serious as this one," Dr. Eric Topol of the Scripps Translational Research Institute tells NPR.

On Monday evening, Hahn tweeted an apology, saying "The criticism is entirely justified."

Hahn tried to explain his error by portraying it as a poor choice of words. "What I should have said better is that the data show a relative risk reduction not an absolute risk reduction."

That explanation is opaque to anyone who hasn't been following the number-twisting saga behind Hahn's claim. Here's the context.

The 35% figure is apparently derived from a study that has not yet been published, that observed tens of thousands of people who were given convalescent plasma to treat their coronavirus infections.

The treatment involves taking blood plasma from people who have previously been infected and transfusing it into sick people. The concept is solid – and has been used to treat other diseases with some success over the past century. These transfusions include antibodies that can neutralize the coronavirus – at least in the laboratory.

But that particular coronavirus study lacked a comparison group – everyone in the study got convalescent plasma. Some people got it earlier in the course of their treatment than others. So to look for signs of efficacy, the study compared the effects in people who received it in the first three days of hospitalization with the results in people who got the treatment later. Researchers then compared the mortality rate of these two groups, both after seven days and after 30 days.

After seven days, the mortality rate of the group that got convalescent plasma early in their disease was about 9%, compared with a 12% mortality rate among people who got it later. At 30 days, the mortality rate was 22% for the early-treatment group versus a 27% mortality rate for the people who got later treatment.

So mortality apparently improved by 3 or 5 percentage points. That's the "absolute" risk reduction to which Hahn's tweet refers. Getting to a 35% "relative" reduction in mortality involves comparing the before and after groups in a way that dramatically exaggerates the effect of the treatment.

Some of Hahn's critics are not satisfied by his apology. "Serious mistakes undermine your credibility," Topol tweeted. "Sorry your [Twitter thread] doesn't cut it," Topol continued. "When will you have a press conference to fully correct this? And fix FDA's website? To restore public trust. Please."

Jeremy Berg, the former editor of Science magazine, chastised Hahn as well. "Be more careful with your language next time," he wrote on Twitter. "This really matters for the public's understanding of COVID-19 and of science."

Adding to the political complexion of this incident, the FDA's press release on Sunday strayed from the usual just-the-facts explanation and included in a headline that the authorization is "Another Achievement in Administration's Fight Against Pandemic."

On top of all this, other critics have questioned the accuracy of the unpublished paper that the FDA relied on in authorizing convalescent serum as a treatment. The work is by respected researchers at the Mayo Clinic and Johns Hopkins University, but it has not been through peer review. As part of that process, a reviewer might ask for clarifications — such as whether the people who got prompt treatment were more likely treated in more recent months, for example, when overall care of COVID-19 patients has improved. If that were the case, it could explain the better survival rates of people who got the plasma quickly.

Based on this study and other smaller efforts, the FDA has issued an emergency use authorization for convalescent plasma. That's short of formal approval. It's based strongly on the observation that convalescent plasma appears to be safe and has reasonable enough signs of effectiveness to be included as a treatment for COVID-19. However, the FDA says the treatment still needs to be subject to randomized, controlled trials that are usually required for the approval of a new therapy. Those studies are underway now.

Researchers who are conducting that research hope they will still be able to attract patients — and have enough blood plasma available – to conclude their studies.

Hahn has previously mischaracterized scientific findings to support Trump administration talking points. In March, he touted a study out of China that he told NPR's Morning Edition showed coronavirus tests developed in that country were of poor quality. In fact, as NPR reported, he misrepresented the paper's findings, based on a cryptically worded scientific abstract that summarized the paper. The actual research had in fact already been retracted.

The FDA's credibility is crucial right now as it evaluates potential vaccines for the coronavirus. Trump has frequently suggested that a vaccine could be approved before the November election. But the public already looks to be wary of a coronavirus vaccine — with a third of those surveyed telling pollsters they wouldn't take it. That position could harden, or worsen, if the public perceives that the FDA has approved a product based on politics, not science.

"When politics mixes with science, as is evidenced here," Topol says, "that could be the precursor to an even worse situation with the vaccine in imminent weeks."

You can contact NPR science correspondent Richard Harris at rharris@npr.org.

Copyright 2020 NPR. To see more, visit https://www.npr.org.

Tags: convalescent plasma  stephen hahn  coronavirus pandemic  FDA  Food and Drug Administration  Policy-ish  Series: The Coronavirus Crisis