Effectiveness of Little Children, Big Challenges: General Resilience Toolkit: School-Based Study

This report describes a study carried out to determine the effectiveness of Sesame Workshop's multimedia educator's toolkit Little Children, Big Challenges (LCBC), which is distributed in the United States (US). It focuses on the English and Spanish versions of the LCBC general resilience toolkit (1) for educators, which is designed to: (i) provide teachers with tools and the language to help children understand and cope with challenging situations at an age-appropriate level, (ii) aid children in expressing feelings around challenging issues, (iii) help children expand their vocabulary around feelings, and (iv) foster the development of emotional competence and literacy of children in terms of emotional understanding and emotional regulation, as well as interpersonal problem-solving skills. This study focuses on: circle of care (attachment); emotional knowledge, understanding, and expression (emotional literacy); and problem-solving skills. As detailed in the report, LCBC is based on The Theory of Change (see Diagram 1 on page 16 of the report) and revolves around partnerships with researchers, experts, community organisations, educators, and service providers.

Participants in this study included 3,245 civilian and military preschool children in San Diego County, California, US. One hundred fifty-seven teachers participated in the study, and 678 parents completed the parent questionnaires. School sites were randomly assigned to active control group (Healthy Habits for Life, a resource kit promoting health and nutrition) and intervention (LCBC) group. Teachers completed a one-day training workshop and implemented the educator's toolkit daily activities (10-25 minutes/day) for 12 weeks. For military classrooms, teachers implemented the classroom activities for 10 weeks. A longitudinal (repeated measures) hierarchical linear model (HLM) was used. Findings are based on the direct testing of children's skills as well as on standardised rating scales completed by teachers and parents. For findings that were found to be significant, an effect size (Cohen's d, which is a measure of the distance between two means, measured in standard deviations) was calculated to determine how large the effect was for the intervention. For research in education and psychology, the typical effect size ranges from .20 to .25, which is classified as a small but meaningful effect size in these fields of study.

Selected findings from the study include:

In terms of "protective factors":

• There was a "significant increase" in attachment relationships (teacher-child) in intervention classrooms, as reported by teachers. There was no increase found for active control classrooms. In addition, there was a "significant decrease" in teacher-child conflicts in intervention classrooms when compared to active control classrooms, but the effect was small (d =-.186).
• There was no significant increase in parent-child attachment relations in active control and intervention classrooms, as reported by parents. Parent-child attachment increased in military intervention classrooms but declined in military active control classrooms.
• Children in intervention classrooms had "significantly improved" in their ability to regulate their emotions in the classroom, as reported by their teachers. There was a significant decrease of "lability" (something that is constantly undergoing change or something that is likely to undergo change) and negative emotionality in intervention classrooms, unlike active control classrooms.

"Emotional literacy" is defined here as the ability to identify 4 basic emotions - happy, sad, mad, and just okay - and to understand that people's emotional reactions depend on their desires. Individual direct assessments of children showed that those in active control classrooms "significantly improved" their ability to understand how external causes affect the emotions of others, understand that a person's belief will determine his or her emotional reaction to a situation, and understand that memories can affect emotions. Children in intervention classrooms demonstrated a "significant increase" in their receptive and expressive emotion vocabulary skills (e.g., verbally recalling and expressing emotion words learned in the classroom). They also demonstrated a "significant increase" in their ability to discriminate between and within emotions, receptively. Cohen's d values ranged from .32 to .91, showing a moderate to large effect of the intervention. No significant growth was found for active control classrooms.

With regard to problem-solving skills, individual direct assessments of children led to the conclusion that, for intervention classrooms, there was a "significant increase" in children's ability to identify emotions and respond "prosocially" to challenging situations presented as hypothetical peer provocation scenarios that illustrate bullying, teasing, fighting, physical aggression, and relational aggression. No significant growth was found for active control classrooms. Furthermore, children in intervention classrooms demonstrated a "significant increase" in their ability to express the steps of problem-solving ("Breathe, Think, Do") when faced with a challenging situation.

In terms of "resilience outcomes" (an excerpt from the report follows):

• "Positive Behaviors
  1. Children in intervention classrooms showed a significant improvement in initiative as reported by teachers. No significant growth was found for active control classrooms.
  2. Children intervention classrooms showed a significant improvement in self-control as reported by teachers (d =.38). No significant growth was found for active control classrooms.
• Negative Behaviors
  1. As reported by teachers, children in intervention classrooms showed significantly lower attention problems, emotional control problems, and total behavioral concerns. These declines were not seen in children in active control classrooms.
  2. As reported by teachers, the intervention may have prevented the increase of depression of military children, while depression significantly increased in military active control classrooms.
  3. There was a significant increase in somatization [a tendency to experience and communicate psychological distress in the form of physical symptoms] in general active control classrooms, but this increase was not found in intervention classrooms. For the [children whose families are in the] military, there was a significant decrease in somatization in active control classrooms, but no change was found in intervention classrooms.
  4. Military children in intervention classrooms showed a significant decrease in hyperactivity, but the effect size shows low practical significance for the intervention effect...(d = -.117)...
• Social Competence
  1. Social skills in intervention classrooms were significantly better than children in active control classrooms. This was also true for military children in intervention classrooms.
  2. Children in intervention classrooms showed a significant increase in functional communication, while active control classrooms did not.
  3. There was a significant increase in children's adaptability in intervention classrooms, while there were no changes in active control classrooms."

Educator's Guide feedback includes:

• "Teacher Feedback:
  1. Teacher ratings from pre to post-intervention indicated that children in intervention classrooms used emotion words more frequently in the classroom than active control classrooms. Children in active control classrooms used fruits and vegetables and other health-related words more frequently than intervention classrooms.
  2. Teachers in intervention classrooms reported that they felt more comfortable talking with children about their emotions than teachers from active control classrooms, who showed a decline in comfort from pre to post.
  3. Intervention teachers increased in their agreement that, on average, their children know emotion vocabulary words, initiate problem solving, know the steps to problem solving, and are able to ask for help when needed. These increases were less dramatic for the teachers of active control classrooms.
  4. Intervention teachers reported that they have observed greater increases in children's ability to recognize emotions in self and others, use of problem-solving skills, and use of learned emotion words to describe others feelings and their own feelings. These increases from pre to post were much less for active control teachers.
  5. Intervention teachers reported that children used 'breathing deeply' more often as a way to resolve conflicts in the classroom than children in active control classrooms.
• Parent Feedback:
  1. 87% of intervention parents reported that the intervention had a positive impact on their child. Moreover, of these parents, almost 50% reported that the intervention had a 'great impact.'
  2. Parents of children in the intervention reported increased use of problem-solving words, determined and persistent, while the control group changed more in their use of 'try again.'...
  3. Compared to active control parents from pre to post, negative strategies of yelling/screaming, withdrawing, responding aggressively, getting angry, and crying all declined according to parents in the intervention group. The positive strategies of talking it through, breathing deeply, and asking for help, all increased among parents of intervention group children.
  4. Parents of the intervention classrooms reported an increase in the following statements. Parents of the active control classrooms, however, declined in their agreement. Statements include:
     • It's important for me to incorporate physical activity into my child's daily routine.
     • My child tries to solve problems by his/herself before asking for help.
     • My child is able to ask me for help when needed.
     • Uses feeling words other than happy, sad, mad and scared.
     • My child uses words to express their feelings.
     • I feel comfortable talking to my child about their feelings.
     • It's important for me to provide opportunities for my child to learn ways to handle conflicts with others.
     • It's important for me to provide opportunities for my child to explore their emotions."

Intervention teacher feedback about the toolkit showed findings such as the fact that 86% of teachers reported that they have heard from parents that children are talking more about emotions and expressing their emotions more at home. Findings also indicate that 69% of teachers agreed that the toolkit increased how much children talked about emotions and feelings outside of Sesame Street Time. When asked how much the toolkit increased children's ability to resolve conflicts constructively, 96% of teachers felt that the toolkit increased children's ability. When asked how much of an overall impact teachers felt the toolkit would have on children's ability to handle challenges in everyday life, 90% of teachers indicated that the toolkit had an impact on their ability. Teachers also felt the toolkit would have an impact on how much resilience-building activities they will do in their classrooms going forward.

Click here for the 104-page report in PDF format.

1 Additional LCBC resilience toolkits were also created for children coping with divorce and an incarcerated parent. Separate evaluation reports were conducted for each of these toolkits and are available upon request.