Registration of Trials: A Census

Apologies to the millions of avid followers of Lost in Translation for the long haitus.  In response to an international petition campaign, with several Nobelist signatories, I am cautiously restarting this blog with the aim of (monthly??) blogposts on troubles and turmoil in clinical translation.

We lead off with an article in this week's JAMA, led by Robert Califf, which provides a census of the clinical research enterprise through an analysis of registered trials at clinicaltrials.gov.  As with many such surveys of clinicaltrials.gov, the picture w/ respect to registry compliance ain't pretty.  Some particularly troubling highlights: the proportion of trials that were registered AFTER beginning enrollment was 52% between Oct 2007 and Sept 2010, and 6.8% of trials do not report their primary purpose (as required).

More generally, Califf et al finds 62% of registered trials are drug trials (the remainder involve procedures, diet, etc.); 63% involve North American research sites; 32% are industry sponsored; 15% are phase 3.

There are some interesting tidbits buried here.  For instance, many commentators are critical of phase 4 studies- viewing many such studies as trials aimed primarily at marketing (phase 4 trials test drugs that have already received regulatory approval for marketing).  Califf et al find that phase 4 studies are significantly less likely to report using blinding compared with phase 3.

The commentary by Dickersin and Rennie makes for a riveting read for those interested in the broader clinical research enterprise.  (photo credit: D. Clow 2008)

Tea Leaves: Predicting Risk and Benefit in Translation

Every early phase trial begins with a series of predictions: that a new drug will show clinical utility down to road, that risks to study volunteers will be manageable, and perhaps, that patients in trials will benefit. Make a bad prediction here, and people potentially get hurt and resources wasted. So how good a job do we do with these predictions?

Hard to know, but given the high rate of failure in clinical translation, there are grounds for believing that various stakeholders go into early phase trials with an excess of optimism. In the current issue of PLoS Medicine, Alex London and I posit two problems with the way decision-makers make predictions in early phase trials. First, they underattend frequent and systematic flaws in the preclinical evidence base. Secondly, they draw on an overly narrow evidence base (what we call "evidential conservatism") that obscures an assessment of whether preclinical studies in a given research area are a reliable indicator of agent promise.

As an open access journal, readers are invited to view our article here. The article has garnered a decent amount of press- digestible summaries can also be found at the Scientist and Pittsburgh Gazette. Also check out a commentary commissioned by the journal editors. (photo credit: canopic 2010)

Dirty Windows of Drug Development

Think of clinical trial data as a window on the efficacy and safety of a drug. Think of data protection and trade secrecy as soot. The above picture? This is the public view on drug safety and efficacy.

According to a recent report in Nature Biotechnology (Feb 2011), medicine may be getting some soapy water and a squeegee, thanks to several policy initiatives at drug regulatory authorities. In Europe, the main drug regulatory authority, EMA, recently issued a policy that will make publicly available "full clinical trial reports"-- even for drugs that are not approved for licensure.

The reforms roughly parallel a series of proposed policies at FDA under the FDA Transparency Initiative. Among the proposed items that would be publicly accessible: when an application has been submitted to the agency (or withdrawn); whether a significant safety issue triggered withdrawal, and reasons why the agency turned down an application.

Disclosure of such information carries some risk. Contrary to common belief, information disclosure does not level all power and influence, as some parties are better equipped to aggregate, analyze, and act on information. No doubt, such transparency will be used by various parties to harangue FDA for otherwise enlightened regulatory decisions.

However, what the public sees of safety and efficacy information- to mix metaphors- is merely the tip of the iceberg. The Nature Biotechnology report, for example, describes the case of Pfizer's SSRI drug Edronax. Published trials included data on 1600 patients, but in actuality, trials involved 4600 patients. When complete data sets were obtained and reviewed, the drug turned out to be no better than placebo, and possibly unsafe (read more here). [[Yet one more reason to wonder what Canadian Institute of Health Research was thinking when it appointed Medical Director of Pfizer Canada to its Governing Council.)]]

Any transparency reforms would provide a much better basis for a) circumventing ethically suspect information practices so that healthcare systems can assess the totality of evidence on drug safety and efficacy, and b) getting a better understanding of the drug development process- warts and all. (photo credit: Lulu Vision 2007).

More Gray Matter: Parkinson's Disease and Gene Transfer

Several groups are pursuing gene transfer strategies against Parkinson's disease. No small task, because for these approaches to work, investigators have to deliver vectors deep inside the brain using surgery. I have previously written that early phase studies using surgical delivery press the boundaries of acceptable risk, because patients can generally manage their disease adequately- though far from completely- with dopamine replacement, and study participation entails nontrivial surgical risks (by my calculations, about 0.5% chance of mortality, depending on the approach).

In the December issue of Lancet Neurology, Marks et al report results of a phase 2, sham controlled trial of CERE-120. The results were negative. That is, for the main measure in the study, improvement in symptoms at 12 months, patients receiving CERE-120 did not do significantly better than patients receiving sham. On the other hand, the product did not raise any major safety issues, apart from a hemorrhage during surgery in one patient.

The team performing the study has emphasized several "positive" outcomes. For one, patients receiving CERE-120 generally responded better than patients in the sham arm (though not significantly better- that is, differences may be attributable to chance). And on a secondary endpoint- response at 18 months- patients receiving CERE-120 did indeed perform significantly better. So did Ceregene score against Parkinson's disease? In an accompanying commentary, French Neurologist Alim Benabid says "the findings... provide the first clinical evidence of a clinical benefit of gene therapy in Parkinson's disease."

I ain't no neurologist, but I say: hold on a minute. When researchers start trials, they pick primary endpoints based on where they think they are most likely to succeed. In this case, the researchers picked improvement at 12 months, rather than at 18 months. From the looks of it, they backed the wrong horse- patients did significantly better at 18 rather than 12 months. What does this tell us? Success in a secondary endpoint might have occurred by chance, and the fact that researchers were unsuccessful on their primary endpoint indicates that they do not yet understand enough about their system to pick the "right" endpoints. So I see this as symptomatic of scientific uncertainty rather than a glimpse of medical destiny. [[One other issue to consider: it is possible that surgery itself (rather than gene transfer) may have caused symptomatic improvements.]]

The study was well reported and provides, yet again, evidence of the utility of sham comparator arms in studies involving Parkinson's disease. One disappointing feature, however, is that the authors did not report whether patients or clinicians could correctly guess their treatment allocation just prior to unblinding. Without this, it is difficult to exclude the possibility that any difference between groups- even at 18 months- was due to "placebo effect." (photo credit: Vin6, 2007)

Icarus, again: Adversity in another Gene Transfer Trial

Two weeks ago brought good news and bad news for gene transfer. First the good news. New England Journal of Medicine beatified a new gene transfer strategy for Wiskott-Aldrich Syndrome (WAS). WAS is a primary immunodeficiency that primarily affects boys. It is thus in the same family of disorders that have been, in varying degrees, successfully addressed using retroviral gene transfer. Like other immunodeficiencies, this represents relatively low hanging fruit for an approach like gene transfer, because scientists can access and target stem cells, and because corrected cells should be at a selective advantage for survival compared with uncorrected cells.

The NEJM article reported clinical, functional, and molecular outcomes for two boys in a trial based in Germany. Briefly the two boys were given a type of chemotherapy (in order to make space for genetically corrected cells), and then transplanted with “corrected” blood stem cells. The corrected blood stem cells contained a viral vector similar to those used in previous gene transfer trials of primary immune deficiency. The team saw: 1) stable levels of genetically corrected stem cells that expressed the WAS protein (indicating the genetically modified cells “took,” and produced WAS; 2) recovery of the function of a variety of immune cells; 3) reduction of disease symptoms, including improvement of eczema, and reduced severity of infections.

The article exhaustively ruled out events that have occurred in other, similar gene transfer trials in which children developed leukemias from the vector. Now the bad news. The same day NEJM published the results, American Society of Gene and Cell Therapy (the largest professional society devoted to gene transfer) released a statement saying that the German team just announced “a serious adverse event in a gene therapy trial for Wiskott-Aldrich syndrome (WAS)”- one of the ten children in the German trial developed a leukemia.

And so continues the saga of gene transfer: three steps forward, one back. (photo credit: vk-red 2009)

Are Trials Necessary?

Today's New York Times ran a heartbreaking story by Amy Harmon about two cousins who developed melanoma. One was entered into a cancer clinical trial and received the investigational drug PLX4032. The other was ineligible for the trial, and therefore unable to access the experimental drug. Guess which cousin died?

The article is one in a series of Harmon articles that seems to raise questions about whether rules governing drug testing and research are depriving desperately ill patients timely access to curative therapy. In this article, the narrative takes aim at two practices: 1- the practice of including control groups within trials, and randomly determining that some patients will receive standard care that is widely regarded as inadequate; 2- excluding patient access to drugs that have not yet demonstrated unequivocal therapeutic advantage.

As with many of my blog entries, I preface this one by saying that I am not a cancer doc, and therefore not in a position to evaluate whether PLX4032 is the wonder drug this story makes it out to be. I also preface my comment on this article by acknowledging the incredible pain and anxiety that patients suffer when denied access to a trial, or when denied access to a preferred drug within a trial. These disclaimers aside, I found the tenor of this article very problematic.

First, the reason investigators randomly determine treatment choice in trials is because, at the outside of a well designed trial, there is genuine uncertainty about whether the new drug is better, the same, or worse than the (inadequate) standard treatment. Many doctors participate in trials because they fervently believe the new regimen is better than the standard one. But the evidence shows, again and again, that on average, new drugs outperform old ones in a small portion of instances (maybe around 15-20%). It is just as likely that new drugs will underperform standard treatments- making patients sicker perhaps, or failing to deliver as much punch. So one concern about the article is the premise that doctor's personal beliefs about which cancer drug will perform better in a randomized controlled trial carries some moral weight. The evidence shows doctors in the aggregate haven't a clue- which is why functional healthcare systems run trials.

A second troubling premise here is that there is no harm to allowing public consumption of drugs that are not yet validated in rigorous clinical trials. CEOs of many pharmaceutical companies perhaps may share this view. But the historical record shows otherwise: in fact, many patients are severely harmed when drugs are introduced into clinical use before they have been established as safe and effective. Perhaps a few readers out there may be familiar with thalidomide? Or autologous bone marrow transplantation for breast cancer? Ever considered the price tag on these new cancer drugs, and do you want your government or insurance company purchasing a potentially useless drug?

Still, article zeros in on an ethical tension that is very difficult to eradicate from clinical research. Patients want- and are entitled- to be treated as individuals. Physicians also prefer to treat patients as individuals. Clinical trials, however require that patients be treated as tokens of larger populations- that they be treated, in a sense, as "stand ins" for future patients. Randomization has not been shown to deprive patients of access to life preserving drugs. However, it does rob patients of fulfilling their desire to be treated as individuals and to exercise personal choice. And this is one of the reasons why the field of research ethics is endlessly fascinating, important, and nettlesome. (photo credit: travelingMango 2008).

Embryonic Stem Cell Trials Start Development

So, FDA has lifted a hold on the first ever clinical trial testing cells derived from human embyonic stem cells. The study- based in California and sponsored by the biotechnology company Geron (view press release here)- will administer cells derived from human embryos ("neural support cells") to ten patients with recent spinal cord injury with the primary aim of demonstrating safety. The same study had been initiated last year- but halted after safety concerns arose in rodent tests. According to news reports, Geron was able to deliver a clean package of studies to FDA. And so- roughly 20 years since the first ever gene transfer study, we now have the first ever human trial of embryo stem cell derived tissue.

Circumstances surrounding this study show all the signs of repeating phenomena surrounding other, highly publicized first in human trials. My book picks up on these signs as harbingers of challenge. Take, for example, the way the initiation of the study itself is seen as a milestone- as a visible sign of medical progress. This presents a regulatory decision as a stand in for study results; it confuses desire with results. And so, I worry when people like Alan Trounson, president of California Institute of Regenerative Medicine, say things like "I think it's a very important milestone for the whole industry... It's very important that they get on and treat the patients...." I also worry when I hear that share prices in Geron rose by 17% on news of the decision- as if permission to study a product is evidence of the product's promise.

The New York Times article reports that Advanced Cell Technology is seeking permission to test another embryonic stem cell derived tissue in human beings- this time for an eye condition. Approval of Geron's trial will surely blaze a path for the latter- though the ethical justification for proceeding in such research may prove more difficult than for spinal cord injury

(photo credit: Brennan G. Wills, 2010)