So one of the mantras within PBIS is that you have this triangle. The triangle fits with those of you who are interested in response to intervention or multi-tiered systems or systems of care structures. It’s borrowed from the community health literature. So for those of you from the health environment, this is your idea cast onto education.

The basic idea is always start with prevention. Look first at primary prevention. Implement an effective intervention, whether it be reading, math, writing, behavior, communication, whatever it is, it should be something that is extraordinarily efficient and effective that will work successfully with at least 80 percent of your population.

Focus first on prevention. In terms of behavior, this is a giant message, because one of the things that you’ve seen – if you’ve worked at all in schools – is that everybody shows up and we expect the kids to behave well. We wait for them to foul up, and then we hammer them. Right? Focus on prevention first. Build that foundation.

In the old days we would come up with these programs and we’d manualize them. We’d put them out there and we expected everyone to turn to page 9 and we would say on page 9, this is what you do. It worked great – for about 30 percent of the situations. What we left people with is this: What do you do when Elliot doesn’t do it right?

And here is what I really want you to take away: Scaling up means focusing on all.

It means focusing on high-quality education with equity for all. For kids who have difficulty learning. For kids from different cultures. For kids who don’t speak the same language. For kids who come from environments that break your heart. If you’re going to build strategies that actually work for America’s education system, you’ve got to build strategies that are going to have large-scale application.

Don’t give us something that only works for a few people. Tell me how it works for everybody, but then tell me how I get it to work for the 15 percent who – they don’t need bazooka-level interventions, but they need a little more structure. They need a little more frequency.

For example, with literacy. The green part of the triangle is a phonics-based program with explicit instruction for 90 minutes a day, for a minimum of four days a week with universal screening, progress monitoring, and effective early intervention. That’s the green. The yellow part for literacy is an increase of about half hour a day, decrease in the size of the group, increase in the frequency with which you get feedback. We can do this. Now we actually have tertiary systems. We can do that with behavior. We can do it with math. We can do it with reading. We can do it with communication, too, but we need your help.

Then, there will always be three to five percent of kids whose primary mission in life is to help us maintain our humility. Those are the kids who are actually going to teach us best.

One of the things that I’ve learned is good teachers really learn when they pay attention to the kids. Kids who have difficulty learning are the best instructors of high quality teaching. One of the banes in education are really smart kids who will learn almost any way you teach them, and then you take advantage of it even though you teach poorly. Everybody learns when you teach well. 

The PBIS Triangle: Lessons

Making PBIS Work: Effective, Equitable, Efficient

In essence, here are the three things I want you to come back to. We talked about efficacy and effectiveness. You’ve got to have the data. If it doesn’t work, then let’s move on. What I’m saying here is getting it to work is necessary but insufficient. It needs to work with equity.

We are abandoning a huge proportion of children in the United States. Forty percent of kids in Oregon don’t graduate from high school in four years. That is abysmal. You want to get kids to communicate, we need to get kids so that they are college and career ready in time. We need kids who have the self-regulation skills, who have the organizational skills, and who have not just the tasks and skills that we’ve taught them, but the ability to learn things that we haven’t taught them.

The other part is, we’ve got to do it with a much greater sense of efficiency. One of the reasons why things are not adopted is we have not paid attention to — not just how much it costs and how much time it takes to adopt them — but how much it costs to actually sustain them. We’ve got to have things that are effective, that are equitable, and that are efficient. We have underestimated equity and efficiency in the things that we’ve built to date.

Research on PBIS

Experimental Research on SWPBIS

One of the things coming here, it’s required that anybody who stands up be able to actually document that what you do is functionally related to change in student outcomes. [See the References section for a list of ten randomized control trials.] I grew up as a single-case researcher. All of these studies are based on single-case methods, and all of these are group design studies. If you don’t get quite through all of them, here’s the punch line.

If you implement school-wide positive behavior support with 80 percent fidelity, here’s what you get.

• You get a 20 to 60 percent reduction in office discipline referrals.

• If you do it over three years, you get a 40 percent increase in the proportion of kids who will meet state standards on reading and math.

• You get an increase in the likelihood that kids will actually both attend school and attend class.

• You’ll get an increase in the proportion of ten-second intervals that they spend paying attention to instruction.

  Turns out, the more time you actually spend paying attention, the more you learn. Who’d have thought? We actually did the study to show that.

• It increases the likelihood that the kids identify the school as a safe environment.

• It decreases bullying behaviors, increases organizational efficiency, decreases staff turnover, and it increases the extent to which teachers perceive themselves as being efficient.

• The most recent study demonstrates a statistically significant effect, with a modest effect size of only about 0.3, but it is a demonstration that kids in schools implementing PBIS actually increase the social-emotional competence with which they leave.

Number of Schools Implementing SWPBIS Since 2000

Now, we’re currently working with nearly 20,000 schools. We actually have PBIS implemented in about 20,000 schools.

School Implementation of PBIS by State

And this is by state. You can’t read this, but this is states in alphabetical order. There are over 1,800 schools in Illinois that are implementing PBIS. There are nearly 1,400 in Florida. There are about 1,200 in North Carolina. There are about 788 schools in California implementing PBIS.

Now, the thing I want you to know is there are 10,045 schools in California. So you think, boy, 744 is a lot. And then you stop and you say well, it’s really not very much.

So you say 20,000 schools. That sounds like a lot. There are 110,000 schools in the United States. Our job is to get to at least half of the schools in the United States before George Sugai retires. We need help.

Lessons Learned in Implementation

Attend to Implementation Science

So given that history, what can I tell you about what we’ve learned? First off, I would like to echo some of the messages from earlier today. We have sat at the feet of Dean Fixsen and Karen Blase. We believe that attending to implementation variables, to the drivers, the stages, the cycles, makes a difference. But part of what we’ve learned also is focus on the systems.

 

And in education, and I’d encourage you to look at this, be careful about the unit at which you are having an impact. Here’s what we’ve learned.

 

In the United States, school districts are the unit that controls education. School boards make decisions. They have funding. And for all the money the state and the Federal Government talk about, it’s trivial compared to what school districts control. You want to change schools, you must change districts. Districts are the unit of implementation, schools are the unit of analyses. The whole school, remember? Change the whole school. And children and their families are always the unit of impact.

 

We’ve got to hold ourselves accountable for what we’re trying to change and where we change it. So within PBIS we no longer work with individual children or schools. We only work with districts.

Active Coordination and the Leadership Team

When we start with a district, we want to see a leadership team. I meet with superintendents from Salinas and San Jose partly because they are looking at implementing PBIS. They have high rates of gang and child violence in their schools. They want to know: What can you do tomorrow to deal with Leonard? And we’ll say we know a lot. We can help you a lot, but we will only do that if you actually change the organizational system and produce high fidelity over a sustained period of time. Never implement something that you don’t expect to last for a minimum of a decade.

So we want a leadership team. Leadership teams always want local demonstrations. The fact that you can do it in Pennsylvania is irrelevant to people in California. The honest truth is the fact you can do it in San Francisco is unrelated to the people in San Diego. Right? So it’s got to be local.

For any leadership team, the thing they want is: Can I give you some schools, and can they be pilots or model demos? The answer always is yes. Yes. But, never implement little, tiny trials unless you build the organizational capacity for sustainability. For how long? Minimum of a decade.

So here’s the deal. You want to have a leadership team. It needs to have these features.

• Can you show me the funding that will allow you to continue focusing on this for a minimum of three years?

• Can you right now tell me how you will communicate with the community, with the families, with the businesses, with the colleagues? Everybody needs to know: what are we doing, is it working, and how is it going to work.

• Do you have the political support? One of the things we look for is if we’re going to come into the district, is the superintendent of the district going to ask how you’re doing at least twice a year? Don’t tell us to come in and help you out unless you’re actually going to build that.

• Do you actually have the policy structure set up so that if we put this in place, it’ll work? I’m going to come back to that.

Now, if the leadership team has that, here’s what we’ve learned to build. You always start with teams in the schools who do things. It’s a blast working with teams in schools. It’s embarrassing that we get paid to do that. But here’s the hard part.

Building active coordination of the leadership team.

• At the district level, always build the training capacity so that when you have done it twice, they can replicate it from then on.You need to always be training trainers.

• Never train trainers unless you train coaches. And I’m going to come back to the coaching part. It is much bigger than what we think.

• If you’re going to do behavioral support, you need behavioral expertise. Language support, math support, reading support. There’s no school in the United States right now that doesn’t have all the resources it needs to implement the green-level implementation of PBIS. No school. Every school in the United States can implement that. But to get the yellow and the red in place, you actually have to have some knowledge. And that’s where you got to build the expertise.

  The same is going to be true with communication. You’ve got things you can teach that everybody could do related to organizational features to build good communication systems. But to get to the kids who have more difficulty, we need to say where do you get that red level and yellow level support?

• Never do this without evaluation. Everybody here, you all are mammoth evaluators. You’ve got, collectively, we’ve got a gazillion units of data right here in this room. Here’s what I can tell you. Most of the data that we have is focused on kid outcomes. To what extent does stuttering occur? To what extent does phonemic segmentation occur? To what extent do behavioral problems occur? To what extent do health education issues occur?

  I want you to add this: Getting kid outcomes without measuring fidelity of implementation is ineffective. I want you to be able to say: Are we doing the intervention? as often as you say: Is it working?

Implementation Drivers

So if we do that, that’s the system. Now, here’s the slide that Renee Boothroyd gave us. And these are the two mentors. This is Dean Fixsen and Karen Blase. Here’s what we’ve learned. I want you to really pay attention.

This is the part of the drivers that focuses on building competence, and I really want you to think about the competence you’re trying to build in regular education and special education teachers to be effective at delivering speech, language, and communication skills. Think about the extent to which you’re trying to get families to assist their children in implementation of speech, hearing, and language skills.

Here’s what we’ve learned: We love training. We love it. We do training like you would not believe. As a group, we are great. We’re actually good trainers as a field. You’re good.

Here’s what we miss.

To what extent do you really focus on teaching people how to select people who are knowledgeable and effective? So for example, one of the things I will train superintendents in tomorrow is: If you’re going to implement PBIS, I want you to build into every job description from custodian to superintendent: are you knowledgeable and skilled at implementation of multi-tiered systems of academic and behavior support. Nobody gets hired without being asked that question. Nobody gets hired without going through the interview process. How do we know if you are skilled not just at green-level strategy, but do you know how to do yellow and red? Do you know how to fit them together? Do you know, as an educator how to work with mental health, juvenile justice, and communication specialists? Do you know how to know if what you’re doing is working?

Second is, never train something unless you coach it in real time, in real place. Coaching is the process of building fluency and adaptation.

• Fluency. I learned something. Can you do it the fast way under real conditions?

• Adaptation. How can you do it with Elaine given that you practiced with Leonard. Because Elaine doesn’t do it quite the same way. How do you adapt and make it fit this environment?

You know we want to focus on having kids show up on time. But you have got to adapt it to the local context. So part of what you’re looking for is how you’re going to make that work across a much larger area. So part of what I would say is, focus on getting all of those drivers in place. Focus on the stages, and focus on the cycles.

SISEP Measures System Capacity

The other part that we have learned is to really focus on measurement. We can now measure not just what happens in classrooms and schools, but the SISEP environment has taught us how to also do capacity of districts and capacity of states to implement evidence-based practices. I encourage you to to go to sisep.org to look at measures of putting that in place.

Focus on Core Features

The last piece I really want to focus on is the extent to which we implement not just practices but core features. We see PBIS as a framework. With practices that have core features that affect outcomes.

Focus on Core Features

Breaking those out, the piece that I would say is important in terms of scaling up: focus not just on the little pieces, but also the overarching pieces that make it work.

Focus on Core Features

One way to think of it is this: If you do these kernels, it frees you up to say there’s not one way to do it, there are many. And we don’t get locked into practices, we get locked into building environments that have those core features. If we do that, we can separate our values (which is what we view as the outcomes) from the science (which is what really works) from the technology of getting that science in place.

Focus on Core Features

When we identify good practices, we need to be much better at not just saying what they are and what do you do. We need to say, for whom is it intended, who needs to be able to implement it, under what situations, and what are the outcomes. We have not described our practices with the level of precision that we’ve needed in the past. If we do that, it will allow us to implement not just by asking if kids are successful, but asking if we’ve actually put it in place.

Focus on Core Features – Example

Think about this. This is a school. Look up here at the data. Think about the green. These are two points in time. The green time and the blue time. You can look up there and say, at the green time, don’t think of all the things you’re not doing right. Pick two. So let’s pick the two circled ones. Now look at the blue time. Did it work? Yes.

Focus on Core Features – Example

But what are you going to do next? We’re going to focus on these additional circles next. You don’t have to do everything all at once. Using fidelity as a measure is a powerful strategy for getting things in place. For 20,000 schools I can tell you that 75 percent are implementing with 80 percent fidelity.

Implement with attention to the kernels, be more flexible about how you actually embed the practices, and over and over again, if you want to implement to scale, implement with the systems that will allow you to sustain things with high fidelity for a minimum of a decade. Thank you.

Audience Question

Can you give us an example of the things that you measure?

So for PBIS, part of the core features are:

• The extent to which you’ve identified three to five positively stated behavioral expectations.

• The extent to which you have an acknowledgement system that acknowledges kids on a regular basis for doing things the right way.

• The extent to which you have a correction system that is not a punishment structure, but is a corrective feature that decreases the likelihood that problem behavior is rewarded.

We can measure that in schools by asking schools a set of questions. We go in and we actually stop the kids. We say kid, do you know the behavioral expectations? Do you know what it means right here? Has anybody acknowledged you for doing things the right way in the past two weeks?

And basically, in schools that are working well is, the kids know the expectations. They can actually tell you what being respectful means wherever they are.

The flip side is, often times we say has anybody acknowledged you for doing things the right way in the past two weeks. And they say: Oh, no. No. Teachers always think they’re much more positive environment than the kids do. But if we get the teachers asking those questions on a regular basis, they figure out – not by things we told them, but by things that they figure out on their own – how to get that accomplishment.

References

Bradshaw, C.P. Koth, C.W., Thornton, L.A. & Leaf, P.J. (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10, 100–115. [Article] [PubMed]

Bradshaw, C.P. Koth, C.W., Bevans, K.B., Ialongo, N & Leaf, P.J. (2008). The impact of school-wide Positive Behavioral Interventions and Supports (PBIS) on the organizational health of elementary schools. School Psychology Quarterly, 23, 462–473. [Article]

Bradshaw, C.P., Mitchell, M.M. & Leaf, P.J. (2010). Examining the effects of School-Wide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. . Journal of Positive Behavior Interventions, 12, 133–148.[Article]

Bradshaw, C.P., Reinke, W. M., Brown, L. D., Bevans, K.B. & Leaf, P.J. (2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education and Treatment of Children, 31, 1–26. [Article]

Bradshaw, C.P., Waasdorp, T.& Leaf, P.J. (in press). Effects of School-wide positive behavioral interventions and supports on child behavior problems and adjustment. Pediatrics.

Horner, R. Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A. & Esperanza, J. (2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133–145. [Article]

Horner, R. Sugai, G. & Anderson, C.M. (2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptionality, 42, 1–14.

Ross, S.W., Endrulat, N.R. & Horner, R. (2012). Adult outcomes of school-wide positive behavior support. Journal of Positive Behavioral Interventions, 14, 118–128. [Article]

Waasdorp, T., Bradshaw, C.P. & Leaf, P.J. (2012). The impact of schoolwide positive behavioral interventions and supports on bullying and peer rejection: a randomized controlled effectiveness trial. Pediatric Adolescent Medicine, 166, 149–156. [Article]