What I’m going to talk about then that is kind of a culmination of some of this, and a representation, are what are called the active implementation frameworks that are part of the work that we do at NIRN.

With the active implementation frameworks, what it suggests is that successful implementation really starts with the what. We need some specificity about the essential ingredients, about what is the what. So that when it’s trying to be replicated, we know what those essential ingredients are and we have an idea of how to measure the extent that they’re present and the strength that they might have.

Successful implementation really does require active use of implementation best practices. These are evidence-informed best practices in terms of some implementation work that’s been done. And we might call those the drivers of implementation.

I already kind of alluded to the idea of purposeful activities that occur within stages of this work.

And a deliberate, purposeful focus on people, or the who of this work, especially in kind of systems and transformational change projects.

Implicit in there is that focus on continuing improvement or improvement cycles.

What: Usable Intervention Criteria

If you pay close attention you realize that there’s a multiplying factor here. So imagine if either one of these are zero, or either one of these is put into place on a level less than optimal, then you’re only getting a percentage of the kind of outcomes that you’re looking for.

Let me start with the what and emphasize that you’d be really surprised at the extent to which these proven interventions are not really defined in enough detail so that they are functional, operational, very specific. So that they’re teachable, that they’re learnable, that they’re measurable, and repeatable. That’s really one of the critical aspects of where implementation has to start.

In some cases, purveyors of certain evidence-based programs, or even in the case of some of the work that I’m doing with the California Department of Child Welfare, where they’re developing an evidence-informed practice model, some of the real work starts with operationally defining the what.

So you can say, what are you seeing, what are people saying and doing, to be able to say, behaviorally, this is what we mean by engagement. Or this is what we might mean by outreach, or different aspects that might be essential functions.

How: Drivers, Improvement Cycles, Enabling Contexts

Now let me move on just to the how and touch briefly on some of those other frameworks that I mentioned.

This slide summarizes that those drivers are factors that are involved in implementation, they pay attention to the person on the left in terms of the competency drivers, that user or that deliverer, if you will, of the intervention or the practice or the kernel.

 

It pays attention to the organization in terms of the degree to which the organization can remove barriers and pay attention to feedback from users, and create data systems to inform decisions that can help have practice inform policy, and have policy then enable practice.

 

And it pays attention to leadership, and by that I don’t mean a philosophical commitment, because we can all say, “Oh yeah, I support that.” I mean taking an active role in paying attention and removing barriers, and being a voice and a message for the kind of change that is being sought. There’s many more elaborations that we can talk about there.

This “how” also continues in a couple of ways with these improvement cycles in terms of some rapid can-do study act approaches. And this really also reinforces this idea, especially with implementation, and there’s probably gradients to this as you think about a specific, very particular strategy, compared to maybe a more multipronged kind of practice approach that you might be starting to maybe implement in a slice of an organization.

 

In other words, we’re not trying to apply this practice model with the California Department of Child Welfare in every county in California all at once. We’re starting with a slice to work out some of the kinks, to try and begin to think about how something like coaching can really fit into an organization that’s very compliance-orientated. Or to begin to incorporate direct observation as a way to measure delivery of the intervention in a system that hasn’t really used direct observation either at all, or it has used it in a punishment kind of way.

 

This just reinforces starting small and using some of those feedback cycles to work out kinks as you then maybe consider scaling up down the road.

This other aspect of using information in an ongoing fashion is just reinforcing, then, that that using data and connecting this kind of information about implementation, about coaching, about the barriers that are getting in the way of rollout, can be informed in an ongoing way.

To influence not only policy…

… But also practice.

All this happens within what we need to have as an enabling context, and I think that’s why we always need to remember that systems trump programs.

 

We can take any effective innovation, and if we put it within a system as-is, it might work for a little while, it might not work at all, but likely it’s not going to work long term. And that’s because these organizations and systems are really designed to, whether it’s intentionally or unwittingly, achieve precisely the results that they get.

 

If we’re not paying attention to how this particular practice model, in this case, influences how a supervisor does their work, and how the organization might need to pay attention to that.

This isn’t saying that the entire system has to start from scratch, but if we start to pay attention to things that are or are not working in service to what we’re trying to put into place, we might be able to change a couple of practices. The return on the investment is huge in terms of really creating a facilitating and enabling context for this kind of work.

 

An effective innovation needs to be supported by that organization, and this active implementation approach creates ways for the system to do that, to pay attention to it in different ways.

Who: Implementation Team(s)

One of the earlier slides really picked up on this idea of who, and we’ve learned that without a deliberate, purposeful focus on implementation, and this often takes the form of starting with a few people, a portion of whose time is repurposed to pay attention to some of this, whether it’s coaching, whether it’s data systems, whether it’s linked communications with leadership groups. So that they can intentionally be building readiness and installing these implementation practices, and really assessing and reporting on outcomes.

How/When: Implementation Stages

And finally, as mentioned in some of the other aspects of the talk so far, this work definitely unfolds in stages. Some of the frameworks name them different things, but essentially we’re talking about some pre-implementation, some initial start-up and during, and then certainly looking towards more long-term, sustained. This often can take the full rollout, it’s not overnight and it can take multiple years. But at the same time, we’re paying attention to specific aspects of implementation as we go.

It’s not as if we are expecting some kind of latent feedback three or four years later to let us know if we’re on the right path.

Example and Final Thoughts

What this one slide just tries to outline is an application of some data that, in this case, was a child wellbeing project.

It was putting into place a lot of those drivers or factors related to implementation around coaching, around ongoing assessments.

And also then measuring fidelity, again in a robust way not in a rigid way, to really look at the percent of child wellbeing cases in a service organization that were meeting some of the definitions of what those essential ingredients were.

Early on, as some of the work was beginning to roll out and get traction and maybe in some cases not get traction at all, and revisiting some of this, with a score with how we were able to measure the rollout of those factors.

You can see that it was lower in terms of getting started to be putting into place, with a number more towards one, and then a lower percentage of those cases.

Over time, as those implementation factors were really being addressed, and in this case training and a real emphasis on training with the 1.5, actually went down a little bit and that’s because coaching really started playing a role.

Implementation is really teaching us that if we don’t have ongoing ways to apply and learn from this in our everyday work, then the return on investment is very low.

So we might have behavioral rehearsals in training, but we need ongoing feedback in an instructional and supportive way to be getting that kind of traction. In this case, they were able to begin to explore some associations between what implementation looked like and what fidelity looked like.

So let me just leave you with a few kind of considerations to take with you. One is that implementation is very much a deliberate and purposeful process, and by default, if it’s working in those real world conditions, then it is multilevel. It involves people, it involves organizations, it involves systems.

The second is that with these implementation strategies and processes, what we’re really looking for are implementation outcomes. It’s that kind of step back, we’re looking for changes in professional’s behaviors, in organizational behavior and systems behaviors. And that those then set the stage, by an intervention being fully and effectively in place, for getting to outcomes. So it goes back to that hybrid or kind of parallel process approach in the design of some of this work.

And I can’t stress enough that we know enough at this point, not to say we’re not going to learn more, I mean this work is going to keep teaching us, but we know enough about what influences implementation to create better conditions, more learning labs if you will, to put some of these practices into place, and to be studying those rather than just going back and saying you know what training, still inefficient, it’s not going to quite get us there. So let’s take what we are learning, and what we have learned from implementation so far, and implement that into our practice to use those as our learning labs for advancing the research.

So in that light, these implementation frameworks, they’re not really an end point, they’re really a new beginning to begin to also take steps forward to understand implementation, best practices, science and policy, and it really suggests for us this idea of starting small and working through some of the roll out of these strategies to effect the kind of change that we really want.

Acknowledgements

So I leave you with a few acknowledgements of some of the work that informs this, and call your attention to that Active Implementation Hub.

Further Reading: Websites of Interest

National Implementation Research Network (NIRN). (2015). National Implementation Research Network. University of North Carolina at Chapel Hill FPG Child Development Institute.

State Implementation and Scaling-up of Evidence-based Practices Center (SISEP) & National Implementation Research Network (NIRN). (2015). Active Implementation Hub. University of North Carolina at Chapel Hill FPG Child Development Institute.