First of all, I think we need to have more partnerships among researchers and school folks. It is very difficult, and I walk both sides of the line. But we are, at our university, successful at engaging partners in our school districts, especially with our doctoral students doing research. We actually get school districts to do randomized controlled studies. Now, to do that, you have to have a real partnership. You just don’t go in with a proposal and hand it to them and ask them to approve it. You co-construct: What is it, what questions do you have?
As an example, I had a doctoral student who was very interested in strategic reading comprehension research with digital texts. Understanding that there are different ways she could have gone, we went to a school where we had someone who we knew was primed. It was a principal who actually had been a literacy coordinator. So we said, “Oh! This is a good bet.” Because she’s going to be pretty savvy about what’s going on in literacy.
But when we went to the school, we asked her to assemble some key players at the school, and we said to them, “This is what this person is interested in doing, but what are your needs? Where are you having the most problems with kids in terms of reading comprehension?” Turns out it was social studies. Not only that, but it also turned out that the social studies team were the most innovative, motivated. And the principal knew, boy if you go to them with this, they’ll be all over it. Again, instead of coming in and saying, okay we’d to do this in science or we’d like to do this in math, we went in and said, “Where would you like to see this kind of a study carried out?” and we worked out the logistics with them. Rather than just delivering a proposal to them, we co-constructed it.
What should investigators understand about conducting research in school systems?
The problem with school research is that it’s messy. It’s very difficult to control all the variables. But again, when you work in partnership with school people, you have a better shot at that. When you ask them to, for example, match kids in a class or randomly assign them, if you give them enough advance notice — and that’s the key. People have to understand how schools operate. For example, you have to know that come February is when they’re starting to think about the schedule for the following fall. You have to be ready to discuss with them at that point what you might want to be doing in the fall. Otherwise it’s too late.
How does this affect the way you plan your program of research?
Historically, the problem we’ve had is we go in with proposals, and we try to get proposals approved. You really need to go in way before that stage. You need to go in and have conversations: What are your needs? What questions do you have? See, that’s the other thing. We have to be asking questions that they want to know the answers to. Otherwise what is their motivation to participate? And there are plenty of questions to be answered in schools. Plenty. Researchers interested in language and literacy and impact on the curriculum, those kinds of things — and several other things, but that’s certainly my interest. There are so many things that resonate with school people. If you explain it in the right way, and you present it in the right way, and if you understand the logistics of schools. You have to understand how they work. Which is why partnerships with people who are actually in the schools is important.
With me, I’m not under the pressure of when the deadlines are for the dissertations and all of that — but certainly my students are. So we do backward design, and we say, okay so when do you have to have your defense? Okay, so how long does it have to be at X, Y, Z? And we work backwards, and that means, “Oh heavens, we’ve gotta start talking to the schools now.” But I know this. So when I work with my students, I prepare them for that sort of timeline orientation.
Yes, it does change the way you think. I’m working with a school district now — have been working for several months — to pave the way for a grant that I’m writing. I’m going to submit to the National Science Foundation for research in disciplinary literacy with science texts. But to lay that groundwork — because the N that we need is very large. So we have to get a buy-in from the entire district and the middle schools in that district. So I’ve been working with them for months and months and months, and we’re just writing the proposal now.
What advice do you have for starting the conversation with a school?
It’s not tabula rasa. You don’t go in and say, “What do you need?” They don’t necessarily always know what they need. You have to set up a framework. Like, “We looked at your data, and we know that some of your students are struggling with reading comprehension on the state tests.” — Of course, the lights go off when you start talking about that. — “And we think we have an intervention that might help your kids. But we would like to talk with you more about what are the kids you’re most concerned about. What grade levels? Let’s look at the data a little bit more. And again, where do you have a pocket of teachers who would be most likely to want to engage in this kind of an enterprise?”
Do your homework. Go in letting them you know something about them. And springing off their data, in terms of achievement and what they’re concerned about. Of course you can also ask them what they’re concerned about. But again, you don’t just say, “What questions do you have?” You have to frame it.
What are some potential roadblocks in collaborations with schools?
One of the biggest roadblocks, if you will, with getting schools on board with research is if they think you are withholding services in any way — especially for kids who have IEPs because obviously you can’t do that. But even kids without IEPs, this whole idea of withholding services or withholding interventions is problematic for schools because their feet are held to the fire in terms of standards. It’s very important. And when we talk to schools, it’s almost the first thing out of our mouth is to say, “We’re not withholding anything.” But what we’re trying to do — in schools it’s not so much that you have control groups, as you have comparison groups. And that is a distinction that’s important.
We’re not trying to look at this intervention versus nothing. We’re trying to look at this intervention versus routing procedures that go on in this school. Example, in the study I was talking about, the vocabulary study. We had a very specific protocol that we had for our vocabulary intervention that was for 90 minutes a week. We worked with the comparison teachers to make sure they delivered what they would ordinarily do in the name of vocabulary for the same amount of time, with the same words. Nobody was withholding anything. They got what they ordinarily would get. We just did something different. That is a much easier sell. It does change the design, though. That’s a key. You have to think of more comparison groups than control groups who get nothing.
You also have to give them enough information. One of the things we have found in several different studies is there is a great deal of naiveté about research. I actually have a horror story about that. It’s a long story, but the bottom line was I was working with a particular middle school and I kept calling about doing fidelity checks for the interventions. These were reading specialists doing the interventions. I kept saying, “I really have to get in to do fidelity checks.” And the person who was the ground coordinator kept going, “We can’t do it this week, we can’t do it this week.” Finally I said, “I have to come in. I have to do fidelity checks.” She said, “Well, we’ve been meaning to tell you. We had a change in schedule, and we had to combine the treatment and control groups.” I said, “Oh. Okay then. So, we don’t actually have a study.” She suspected that was not right. She did suspect. But I think, had they told me about their schedule problem — because I know enough about schools and how they work, I could have sat down with them and said, “Well, could we do this, that, or the other and save the study?” I think that could have happened. But things happen pretty quickly in schools. And the administrator may have been naive and not really understood what that was doing to the study. So that’s an important thing to think about.
Again, with the understanding that they may not be very sophisticated — that’s not their job. But you also have to be careful to be respectful of what they do know. And not come in on a high horse, “I’m the researcher who knows everything.” You have to cultivate relationships and do things for them as well. So when they need someone to come in to talk to a group of teachers about disciplinary literacy, I go. So we have a relationship to begin with.
