This presentation was delivered as part of a workshop session on the application of adaptive design principles to clinical trials in speech-language pathology. Ray Kent provides summary thoughts and comments on the seminar, with additional comments from Sharon Yeatts, who explored adaptive designs in greater depth in her presentation, Adaptive Trial Designs for the Development of Treatment Parameters.

The following is a transcript of the presentation video, edited for clarity.

Types of Adaptive Clinical Designs

I’m going to have just about six slides and they’re very brief and they’re very general. So each one of these addresses a general topic.

The first general topic is adaptive clinical designs considered in toto. That is the great variety of clinical designs and as pointed out in a recent article by Chow and Chang in the Orphanet Journal of Rare Diseases. There are actually three different types of adaptation.

There are prospective adaptations and that’s really the subject of this particular seminar. Talking about the prospective or adaptive designs in which we can consider things like adaptive randomization, early stopping, dropping the losers, sample size re-estimation and various other types of adaptive designs.

We also have concurrent adaptations. These are adaptations that are made to — generally to randomized clinical trials in which the investigators decide whoa, we have to go back to the drawing board. And these concurrent adaptations frequently involve things like conclusionary exclusionary criteria, changing those typically because oh, subject recruitment is not quite what we thought it would be. Let’s relax our criteria a little bit or let’s allow this nor that to happen. Evaluability criteria, the data, those regiment or treatment duration, hypothesis, study endpoints and other.

And the retrospective adaptations would be primarily things like statistical analysis changes. These are made before the database lock or unblinding of the treatment code. So we’re still in the sense, pretty pure with regard to this whole process. But it’s a reconsideration of the statistical analysis.
Now notice that prospective adaptations, these are implemented by study protocol. So adaptive designs are when all the planning is done in advance. It is intended that there will be an adaptive component.

In the case of the concurrent adaptations these were not intended, but they’re implemented by protocol amendments. And we can elaborate that a little bit as we will in a moment.

In the case of the retrospective adaptations, those are implemented by regulatory reviewer’s consensus.

So in each one of these cases there is a particular type of implementation. Now moving to the bottom of this slide this is really an elaboration of implementation or protocol amendments.

There is a very interesting report by Getz and colleagues published in 2011 in Therapeutic Innovation and Regulatory Science. And what they did was to look at a large number of drug trials asking the question how often are these amended and what is the nature of the amendment.

Well, they noticed that nearly 60 percent of all protocols used in clinical trials for new drugs were amended, the great majority are amended. This was surprising to me, 40 percent were amended before the first subject or the first visit. So right away people are saying let’s go back to the drawing board. An average of 2.3 amendments were made per protocol, and each amendment required an average of 6.9 changes. So there is a lot of going back to the drawing board, a lot of reconsideration, a lot of time lost and a lot of head scratching.

A number of these were found to be avoidable. That is, if the investigators had spent a little bit more time considering the design in advance they might have been able to avoid these. But the point of this is simply to say adaptation is a fairly frequently occurring thing. Whether we intend to do it or whether we do not.

Whatever we do, whether it’s adaptive randomization, early stopping etc. There are important constraints on adaptations. And three of those are controlling the type error, type one, error rate. Number two, being sure that the trial has a high probability of answering the research question. We don’t want to sacrifice that. And finally maintaining equipoise which I think is probably best defined as meaning that in the general scientific community there is uncertainty regarding the advantages or disadvantages of a particular treatment. So we’ve not prejudged the matter but we do think it is worthy of exploration.

Why Do Some Clinical Trials Fail?

Now another very broad question, why is it that clinical trials fail?

We’ve had enough clinical trials and certainly in the drug arena but also in behavioral interventions for us to take a step back and see why there are failures and what that suggests to us about more efficient ways of carrying out clinical trials.

First the treatment works for some, but not all. And this is true not just in the behavioral areas but also, for example, in clinical oncology where it is recognized that there are so-called exceptional responders. Some individuals respond very, very well to a medication, much better than the average patient enrolled in these trials.

And although for a long time there was kind of a reflexive rejection or dismissal of any N of 1 study, and most people then would recoil from it saying that is and there is one of those dreaded three syllable words “anecdote” if you report on a single subject you’ve reported anecdotally. But even in areas like clinical oncology there seems to be a, even perhaps grudging acceptance that end of one can be informative.

When we at the ASHA executive board were trying to get evidence based practice going in our field, we approach the people at the Oxford Group to see if they would consider single subject designs as possibly being incorporated somewhere in their levels of evidence. And they politely demurred, but we are seeing now that even in medicine single subject designs are getting perhaps a better receptance than they might have a few years ago.

Perhaps underlying both of those would be the assumption of the universal pathophysiology of a condition. This is probably easier to grant in drugs than it is in behavior. That is, if we assume that Parkinson’s disease is caused by the same pathological mechanisms then all individuals with Parkinson’s disease can be assumed to respond to a drug in a certain way.

But can we make that assumption for childhood apraxia of speech? That there is one universal pathophysiology? Can we assume that spastic dysarthria has one common underlying pathophysiology? Can we assume that specific language impairment has one underlying pathophysiology? In many of the conditions that we’re concerned with we don’t necessarily have a guarantee that we’re satisfying the principle of universal pathophysiology.

And two more three-syllable words that we often dread: Placebo, where people will benefit perhaps because they believe, even if they are in the control arm of an experiment, hey, I’m taking a good drug, I’m getting better, I feel much better this week than last week. Or kind of the reverse of that the nocebo effect which occurs particularly in individuals with degenerative diseases in which they believe that they are going to get worse even if they are in the clinical arm of the study.

Also questions about clinical end points: When should the study be ceased or what is a reasonable stopping point? And we already heard examples of that.

It’s also possible that different treatments have different mechanisms of action, different sources of outcome variability, different windows of optimal effectiveness in the history of a disorder. And that has already been suggested in the preceding talks.

Frequent Barriers to Clinical Trials in CSD

Within the general field of communication disorders where we’re hoping to make sure that we have evidence based practice for all disorders, we have several barriers. One, many of the disorders are of low incidence. It’s not easy to recruit the participants who will be able to satisfy the inclusionary, exclusionary criteria. Not just for the rare or orphan disorders but even for some disorders that seem to occur more frequently. For example childhood apraxia speech. Not necessarily an easy group to recruit.

We also have the problem of heterogeneity of the affected individuals, including diverse and multi-factorial etiologies. Many of the disorders of interest to us, for example stuttering, may have a genetic base for some individuals at least, but that genetic factor somehow is going to be interplaying with various environmental factors which could vary by individual.

So we don’t have simple etiological principles for many of the disorders of interest. We also have a substantial problem with comorbidity and co-occurrence of disorders. In the case of specific language impairment comorbidities have been reported as anywhere from 5 to 15 percent. For stuttering comorbidities have been reported as high as 60 percent, so many individuals who stutter may also have phonological or speech sound disorders, language disorders etc. So these people do not come to us in one nice clean clinical category.

We also have problems of course with limited resources. We don’t have deep pockets. We’re not funded in the main by large companies such as drug companies. We need to seek support wherever we can get it. Certainly NIH but perhaps also some private funding sources. So we have difficulties with funding, patient access and cooperative efforts.

Non-adherence can certainly be a problem. And that occurs with both some of the participating clinicians and also of course with respect to some of the people who are receiving treatments, the patients or clients.

And end point uncertainty another problem, another barrier in our field.

Example: Aphasia

To take just one example and I don’t have time to take a lot of examples so let me just consider one. And this is aphasia.

And you can see to your left on the slide this is taken from the ASHA website. This is the practice portal evidence map and if you haven’t seen it I certainly recommend it to you because it does represent a serious effort by our association to make available to clinicians summaries of the evidence for the various treatments that are being proposed.

So here we have aphasia and under that you see the treatment tab and these are a number of different procedures that have been recommended for intervention of aphasia. These have received various degrees of research, of clinical trials.

One of the most frequently investigated is CILT constraint induced language therapy. And if you go to the practice portal and click on that you’ll see a summary including a review of evidence for the efficacy of constraint induced language therapy. Which generally points out that there is a value of intensity of therapy.

But the point is we have several different types of intervention. These are conceivably useful, or more useful, or less useful I should say, at various points post stroke. They might be beneficial within different types of therapeutic windows and they might even be useful in combination or perhaps used successively. And any of these could be used in conjunction with ongoing pharmaco therapy.

In a recent Cochrane review by Kelly, Brady, and Enderby they looked at speech and language therapy for aphasia following stroke, their assumption you can see in that text box there, the trials randomized small numbers of participants across a range of characteristics including age, time since stroke, severity profiles, interventions and outcome.

People with aphasia come to us with a variety of different profiles. Very, very difficult to establish homogenous clinical groups for research. What they suggested is we need to know for a specific patient groups, not just one large group of people with aphasia, but for specific patient groups: (1) what is the optimum approach? What particular method of intervention?; (2) the frequency, how often should that be done?; (3) the duration of allocation; (4) the format of therapy, for example individual therapy, group therapy or these days even teletherapy, many different variations that could be imagined in the service delivery arena.

So all I’ve tried to point out here is that there is some very, very general issues that I think relate to this idea of clinical adaptive designs. And I’m certainly intrigued but the possibility of using some of the adaptive clinical designs to give us the opportunity to be sensitive to some of these group differences, but also to make more efficient use of the very difficult to recruit subjects. As we heard from Dr. Langmore and Dr. Robbins that is a frequent problem. My guess is that the people who are doing clinical trials in drugs are not the only ones who are having the need for amendments. And I don’t think there has been a study that I know of that indicates how frequently those are made for behavioral interventions, but just judging from what we’ve heard today that’s probably pretty common.

Questions and Discussion

Ray Kent:

One particular question I’d like to ask Dr. Yeatts is, are you aware of particular cases in which the adaptive designs have been used for behavioral interventions? Is there a literature that we can look to to give us guidance?

Sharon Yeatts, Medical University of South Carolina:

Most of my work is in pharmacologic agents, so I can’t point to something specifically but when I first started working in MUSC I did a little bit of work with the addictions group and my guess is that they have made progress by leaps and bounds in terms of adaptive designs.

A lot of their work is in behavior modification therapy for helping these patients. And I would have to check the literature by I think that might be a place to take a peek and see what they’ve been doing in terms of adapting with behavior modifications.

Some of the adaptive designs would be very easily translated into those behaviors. As I said, the dose findings sort of thing where we have to rethink about what the dose means, what the outcome is might take a little bit more time, but the adaptations that you talked about that you mentioned, dropping the losers, things of that sort would be very easily translatable to those sorts of therapies.

Ray Kent:

And the second question I was interested in relates to some of the foregoing discussion. What can we do with conditions in which we have a progressive loss of ability or health? Most of what we’ve talked about assumes that there is some kind of steady state, but many of the conditions of interest to us like progressive neurological diseases, ALS, Parkinson’s disease, we’re not dealing with a stable baseline. We’re dealing with a very predictable decline in ability. And I wonder if there are some designs that may help us look at something like a rate of change that might be sensitive to those kinds of problems.

Sharon Yeatts:

Your second question is a little bit more complicated. The idea of a progressively degenerating condition where we’re trying to assess how that happens, there are more complicated statistical models which allow us to assess how a particular patient changes over time. And we can assess that rate of change for a treatment versus another treatment. But it would definitely make some of the earlier phase modeling designs that I talked about a little bit more complicated. Because really as you suggested, what we’re trying to say is can we make that rate of change less in the treated population, and that’s a little bit harder to establish.

Audience Question:

I’m trying to think how to conceptualize these adaptive designs. It seems to me — and I want to make sure I’m right — that in the adaptive design you’re doing model convergence to find the best likely optimal value of whatever parameter you’re measuring. But that’s separate from confirming that it’s a statistically significant thing.

Sharon Yeatts, Medical University of South Carolina:

So yes…ish. So there are adaptive designs which can be implemented in the confirmatory setting. This last presentation actually highlighted some of those things like sample size re-estimation.

Actually when you think about a phase three clinical trial that does interim analysis for efficacy or utility that’s actually an adaptive design. It’s adapting the protocol to stop sooner than you had anticipated based on what you’re seeing.

But we don’t think about it that way because it has been ingrained in our training for so long. Adaptive to us means new and exciting. That’s not exciting.

So my general feeling is that adaptive designs are most useful as you suggested them to be. In the early phase exploratory setting where you are trying to establish what treatment am I looking at, what sort of effect size am I seeing. Which patients is it going to work best in. That’s where adaptive designs will really give you the most bang for your buck.

When you get to the confirmatory setting, again this is my opinion, you should know the answers to the questions that the adaptive designs are designed to answer.

If you think about it that way, the adaptive designs are helping you learn from the data as you go. By the time you get to the confirmatory setting you’re trying to confirm what you know to be a potential treatment works. You shouldn’t be fiddling with things at that point. But there is a big push as well to meld sort of the phase two and the phase three together into adaptive designs that make this process more efficient.

Panel Comment:

I guess what we’re saying is rather than embarking on a large phase three trial which is a confirmatory science stage, you really need to look at the phase two stage and the phase one stage to make sure that you’re getting the optimal bang for your buck. And these optimal designs will help you in finding dosages and characteristics of delivery of your treatment in different ways.

I should mention that the NIH has a planning grant for clinical trials and it can be used to considerable degree to work with the biostatistician to set up just these kinds of designs, and I think that stage and that mechanism has been relatively underused. There is, it’s an R I forget what the R number is but there is an R number for a planning grant to do just this kind of work. And people don’t use them as much as they should. They really dive into a confirmatory study before they’ve done the work on finding out the optimization of the treatment. And that’s what that planning grant mechanism is for. And so I would really encourage any of you who are thinking of doing clinical trials to look into that mechanism, contact the staff and discuss how you might use that mechanism.

References

Chow, S. & Chang, M. (2008). Adaptive design methods in clinical trials-a review. Orphanet Journal of Rare Diseases, 3(11), 169–90

Getz, K. A., Zuckerman, R., Cropp, A. B., Hindle, A. L., Krauss, R. & Kaitin, K. I. (2011). Measuring the incidence, causes, and repercussions of protocol amendments. Drug Information Journal, 45(3), 265–275

Kelly, H., Brady, M. & Enderby, P. (2010). Speech and language therapy for aphasia following stroke (review) the cochrane library. Cochrane Database of Systematic Reviews, 26, 2013