Assessing Impact, Evaluating Adaptability: A Decade of Radio La Benevolencija in Rwanda, Burundi and the DRC

"...In addition to understanding the aggregate impact of RLB’s [Radio La Benevolencija] programs, this meta-evaluation seeks to investigate what RLB’s work offers to others engaged in this field."

This synthesis and evaluation of the work of Radio La Benevolencija (RLB) in the Great Lakes Region: Rwanda, Burundi, and Democratic Republic of Congo (DRC), from the Center for Advanced Research in Global Communication (formerly Center for Global Communication Studies at the Annenberg School for Communication), University of Pennsylvania, United States, analyses the methodology of the RLB programme model through the lens of its adaptability to other countries and contexts and how the RLB model might be used a prototype for future interventions. It “compares the impact of the project in the three countries, but …also interrogate[s] whether the model is adaptable and suitable as part of a ‘toolkit’ others can employ in their own media projects.”

The research assesses impact using previously documented cumulative changes in knowledge, attitudes, and behaviours of audiences in the three countries.  In addition, it uses in-depth interviews of key informants and the general public. Also it analyses whether programme planning was grounded in existing research and theory, and whether RLB monitoring and evaluation (M&E) was implemented in a way that would allow RLB to appropriately assess outcomes and impact. And finally, it looks at the transference of the model from Rwanda to Burundi and DRC. RLB’s core activities are: edutainment radio dramas for the general population; weekly factual/journalistic radio and television programs for a leadership audience; and practical experiences for the audience created by implementing grassroots activities.

Using Ervin Staub’s "basic needs theory" (1989) that those living under difficult life conditions often find their basic psychological needs unmet and may therefore "attempt to meet their own psychological needs in ways that are destructive", the radio programming attempts to to help victims understand the continuum of violence by first creating a fictional village and set of characters and then shows how their needs lead them to the formation of in-groups, and then to the scapegoating of other groups. In addition, the programme uses Laurie Pearlman’s work on the healing process which looks at the effects of adult trauma and the benefits of talking about traumatic experiences under safe conditions, as well as the importance of trying to show victims how perpetrators might have come to engage in terrible acts of violence - "that when people are scared and lack security, a sense of belonging, and basic material needs, they can become vulnerable to extremist thinking, may help people forgive."

Among the findings, the research shows that there is more evidence of positive impact where the programme has run the longest: Rwanda. One of the challenges is talking about trauma in countries where the concept did not exist. The factual programming has aimed to reach both leadership who might use ethnic tensions for political ends and the general public. "Increasing citizens’ involvement is done in part through call-in segments of the radio programs." Data on listenership shows that the programme is popular in all three countries. The audience reach is 44%in Kigali, Rwanda (2012), 31-34% in DRC (2012), and Burundi (2010).

The grassroots groups share two key forms: "taking part in ‘listener groups’ that discuss the radio program(s), and holding direct trainings of group members who can spread RLB’s messages in their community both indirectly, by acting as role models, and directly, by passing on knowledge." Also activities such as creating civic associations, conducting community debates, performing participatory theater pieces, and holding community mediation activities are components of the project of spreading knowledge through grassroots groups. However, with a less explicit theory of change, this component is more difficult to assess. One evaluation (2009) found that in Rwanda grassroots groups showed more social cohesion and knowledge about the continuum of violence and trauma healing and that trained change agents could recognise symptoms of trauma. However, evaluating specifically what worked has not been possible.

"Overall Findings

• RLB’s interventions in Rwanda, Burundi, and the DRC have achieved significant knowledge, attitude, and behavior changes. Among the most notable of these:
• With respect to gains in knowledge, members of RLB’s audience have gained better understandings of the cycle of violence and methods used by politicians to manipulate audiences.
• With respect to attitudes, members of RLB’s audience have experienced positive attitude changes regarding trust in communities, the importance of dealing with trauma, the dangers of scapegoating, the importance of active bystandership, acceptance of marriage outside one’s own ethnic group, and the importance of understanding complex truths about the past, developing a shared history, and seeking justice.
• With respect to behaviors, members of RLB’s audience became more willing to hear an opposing group’s side of the story, became less willing to automatically cede to authority, become more willing to attend reconciliation activities, and increased discussion of topics presented in RLB programming with friends and family.
• Given the challenges of conceptualizing and implementing peacebuilding activities in conflict affected areas, the amount of research, evaluation, and adaptation that has gone into RLB’s work is noteworthy and unique.
• RLB’s ‘prototype’ for implementation was composed of carefully considered building blocks, rendering its method adaptable in different contexts.
• RLB’s radio drama Entertainment-Education efforts (implementation of story development and incorporation of audience feedback) have been especially effective, in particular in Burundi and Rwanda, reaching approximately 70 to 90% of their target audiences, respectively.
• The programs’ effects are amplified over time."

Among the conclusions is the recommendation that the "Staub continuum [remain] as the backbone of the program, even as other elements are added, and ...that elements of the continuum are embedded in the media strengthening program design, outcome goals, and M&E efforts."