The following is a transcript of the presentation video, edited for clarity. Presentation slides are available for download via the PDF button in the toolbar.
Originally presented at the ASHA Convention (November 2013) as part of the session Development of a Bilingual Test for Spanish-English Children: A Long and Winding Road. Videos in this series are:
Challenges in Assessing Bilingual Populations
(Elizabeth D. Peña, University of Texas at Austin)
(Elizabeth D. Peña, University of Texas at Austin)
Bilingual Phonology Assessment Design
(Brian A. Goldstein, La Salle University)
Bilingual Pragmatics Assessment Design
(Aquiles Iglesias, Temple University)
Bilingual Semantics Assessment Design
(Lisa M. Bedore, University of Texas at Austin)
Bilingual Morphosyntax Assessment Design
(Vera F. Gutierrez-Clellen)
The pragmatics subtest is a little bit unique in several ways. One of them is that this is the only subtest that can be given in any language. It can be given in English, in Spanish, or bilingual.
We really don’t care so much about what form the kid is using, or what language the kid is using. What we’re looking at is their pragmatic skills. It’s also the only subtest that is not scored, and you’ll see this in a minute.
In terms of pragmatics, I hope you all know that pragmatics is using the appropriate message or interpretation in relation to the communicative context. If that means nothing to you, it’s what is your intent, what is the cohesion of your items, are you paying attention to what the other person knows, and so on and so forth.
I think what’s more important is something that Rhea Paul and Norbury mention. That is, “A ‘test’ of pragmatics is almost contradictory in terms. Since pragmatics involves the use of language for real communication, we need to assess it in a more naturalistic context.”
So actually developing a test for pragmatics was a very big challenge. Because creating a test, according to Rhea and Norbury, is insane.
Pragmatics is going to depend on the context and the experience and social demands of the situation. One of the things that’s interesting, and you’ll see why we don’t score the pragmatics subsection, is that pragmatic disorders are not unique to children with language disorders. So you can have kids with morophosyntactical problems that do not show pragmatic difficulties. Kids with semantic problems, phonological problems, that don’t demonstrate these pragmatics. One of the things that Brice and Montgomery have looked at also, in terms of L2 learners, is that they might encounter some difficulties, and I’ll address those difficulties that these children might encounter.
Instruments and Approaches
There are very few available instruments to assess pragmatics.
Some of them are observation of parent-child interaction, and you’re supposed to be marking what the interaction is like in terms of the pragmatic skills.
There are also questionnaires. Parents are required to fill out a questionnaire with items like, “Talks to people without any encouragement or starts conversations with strangers.” You know, does the kid initiate and so on and so forth.
Or they set up situations like, “Mary saw a little boy stealing candy at the store. She ran over to her mother and said, …” What is the kid supposed to say here? I think this is a very interesting part of pragmatics. It really depends on your experiences, and what’s acceptable and not acceptable. If you’re coming from a middle-class Anglo perspective, you say “Mother, somebody was stealing something.” Anything else would be totally unacceptable.
So, one of the things that also has been done is common, structured activity with temptations. These are things that the kid has done, within the child’s experience, and create a need for the kid to say or do something.
All of us are familiar with Nancy Creaghead’s making a peanut butter sandwich. Well, how many allergies do we have right now? We can’t be doing peanut butter at all. Plus, those of you who know about making a peanut butter sandwich, you know how messy it is, then somebody has to eat that. And the kid usually gives it to me, and I’m going, “Oh, no way. I’m not going to eat this.”
So we created something called, “A present for Diego.”
We focus on communicative intentions, primarily assertive acts. We look at requests for information, like, “What is that called?” Requests for action or attention, “Gimme that!” Requests for clarification, “¿Un qué?” / “A what?” or assertive comments like “This is a fun game.”
Test Example
So, what you actually will see is the whole test.
These are the objects you need. These are things that are in your closet. You need a box, wrapping paper, three short colored ribbons — and you’ll see why. They don’t have to be red, blue, and green. They can be black, orange, and purple if you want. Three long colored ribbons (not long enough to go around the box). Two tape dispensers — one of which is empty. A Mushki. Then we need a puppet or a doll, and this is optional, and you’ll see why we need it.
Looking at the score sheet, you will see it says that this is an optional activity that can be used to establish rapport with the child, as well as to observe how the child uses/understands pragmatic language. This activity is to be used for descriptive purposes only and does NOT contribute to any score.
What you have then is the stimulus on the left-hand-side, the expected on the right, and then whether the kid uses it or not.
So, the examiner says, “Let’s wrap Diego’s present. I bought Diego a great gift. It’s in the box. What do you think it is?”
There’s another alternative that we created. We realized — not our Puerto Rican kids or our Dominican kids, but our Mexican-American kids sometimes were not confrontational to an adult. As you can see, sometimes you have to be confrontational. So we created this puppet. So the puppet is the one these kids can say things to, not the person. So, using this option we would say, “Let’s wrap Diego’s present. This is my friend Timmy. He is going to help us wrap Diego’s present. Sometimes he can be silly, but don’t let him fool you. I bought Diego a great gift. It’s in the box…”
What you’re getting then is this responsive act, “A toy.” Or no response.
So you say, “It’s a Mushki!”
You’re expecting a request for clarification or a request for information. Again, we’re giving the kid options. You really don’t know what the kid is going to do, whether it’s a request for clarification or a request for information. “A what?” / “What is a Mushki?” / Or he gives you the puzzled look.
You say, “A Mushki, see.” Then you open the box. “We use Mushkis to bingle the waddles.”
So, the kid is going to say, “What?” Or he is going to say, “You’re crazy.”
You say, “Let’s see, to wrap the present, we need wrapping paper, tape and ribbon. Do you remember how to wrap a present? Hmm. Tell me.”
And the kid might say, “Bueno, lo primero que se hace es comprar todos los materiales necesarios para completar la tarea.” For those of you who speak Spanish — wow.
Or kid 2 says, “Box y ribbon pa el party.”
Folks, realize one of the things that’s very critical — I’m going to mention it later on. You are picking up a tremendous amount of information with respect to the kid’s language skills in this process.
So, we place the paper and the box on the table. And you say, “Let me have the ribbon.” And the ribbon is not on the table. So the kid has to say, “We only have paper and a box.” Or “There is no ribbon.” Or the kid will say something like, “Where is the ribbon?” Again, it’s assertive acts.
“Oh, I forgot, it’s in the bag.” And you pull out the red, blue, and green ribbon. Now, there were short ribbons and long ribbons. Because the kids might not know colors. And we want to make sure we don’t get into this labeling problem, so we have these three ribbons that he can choose. So you say, “I have a red ribbon, a blue ribbon, and a green ribbon. Which one do you want?” And he points to that one.
And I say, “Here it is.” And the kid says, “That one is green.” Or “That is not the one I want.” Or “Where is the red one?” Again, assertive acts.
“Oh, I’m sorry.” Then you give him the correct ribbon.
You bring up the empty tape dispenser. Which all of you have empty tape dispensers, as you work in the schools, and you begin wrapping the present.
And you say to the kid, “Please give me some tape.” Well, the tape dispenser has no tape on it. So you’re forcing the kid now to say something. “There is no tape.” or “Where is the tape?” or something to that effect.
“Oh, I better get another one.” Then you get the dispenser with tape. “Ok, now we have to put the ribbon.” But you keep the ribbon. And the kid says, “Let me have it.” A request for action. Or, “I don’t have the ribbon.” Or, “Where’s mine?” Again, we’re not really caring about the form, we’re caring about the use. And particularly, assertive acts.
So, you give the ribbon, but the ribbon is too short. You have to measure ahead of time, by the way. You might get these statements from these kids: “The ribbon is too short.” Or, “Do you have a longer ribbon?”
You say, “So what do you think we should do?” This is a request for information. So, the kid’s got to answer. The kids come up with great answers to this. “Not put any ribbons.” “Tape the ribbon.” “Tie two ribbons to make a long one.”
“Great idea. I think Diego is going to love this present. What do you think Diego is going to say?”
“Thank you for the present.” / “I think he is going to hate it.”
So, that is the pragmatics subtest. Again, it provides the context, the expected responses, and then you circle it yes and no.
Interpretation
Remember what I said to you — pragmatics skills are not necessarily tied to some of the morphosyntax or phonology scores, and this is very well proven by our data. The correlation between English phonology, English morphosyntax, and English semantics with pragmatics is very, very low. The same thing happens with Spanish.
What I think we need to look at is these language profiles. If the kid has morphosyntax OK, semantics OK, pragmatics OK, great. If a kid has OK morphosyntax, OK semantics, but low pragmatics: what kind of kids are these going to be? These are the kids that most of the teachers — and we have a lot of data of seeing this a lot — these kids who are in these classes, and their language skill, morphosyntax and semantics are okay, but they’re just sitting there not being assertive in the least bit. And the teacher thinks that there’s a language problem there.
Or you can have a low morphosyntax, low semantics, and OK pragmatics. This is the kid that we know that at least from a use perspective, we’re going to be able to build on that use in order to increase morphosyntax and semantics.
And finally we have these low, low, low kids. Which, you know what they mean.
So, what do we know, and what have we accomplished by giving this subtest?
And also it has provided you with information on pragmatic strengths and challenges.