Complexity, Context and USAID-Supported Social and Behavior Change Programming in Ethiopia 2015-2021

"...complexity arises from context, and every context is different. It is unlikely that one thing will work for all geographical areas, communities, and implementers."

This report shares the findings of an assessment of United States Agency for International Development (USAID)-funded programmes in Ethiopia to consolidate learnings from current and past social and behaviour change (SBC) investments. Conducted in 2022 by Agency for All, the goal of the assessment was to develop a set of evidence-based and actionable recommendations for strengthening future SBC programming in Ethiopia. The research approach reflects a shift in the field of SBC toward a more context- and complexity-sensitive orientation that seeks to provide greater flexibility and put a heavier emphasis on activities' ability to adapt to changing circumstances.

As explained in the report, "Over the past decade, there has been considerable emphasis on social and behavior change (SBC) in Ethiopia. For example, USAID/Ethiopia invested in standalone SBC projects, Communication for Health (C4H) from 2015-2020 and Breakthrough ACTION from 2021-2022. In addition to these initiatives, USAID/Ethiopia has supported SBC programming in Transform Primary Health Care (TPHC), Transform Health in Developing Regions (THDR), Growth through Nutrition (GTN), PEPFAR Orphan and Vulnerable Children, Tuberculosis (TB), and COVID-19. These investments focused on increasing demand for treatment-seeking and service uptake behaviors for maternal, newborn and child health (MNCH), family planning/sexual and reproductive health (FP/SRH), malaria, nutrition, TB, HIV and zoonotic diseases. All of these projects contributed to improved demand for quality health services. Examples of improvements include increases in early antenatal care (ANC) health seeking, early postnatal care, increased rates of exclusive breastfeeding, improved fever treatment for children under age 5, and more. However, despite this progress and significant USAID investment, health service utilization in Ethiopia is still relatively low. For example, the 2019 national Health Extension Program (HEP) assessment showed that 41 percent of community members were not aware of the services available at their nearby health post. Many other assessment findings highlighted weaknesses in service provision and poor uptake of health behaviors by providers." USAID commissioned this study to address these shortcomings and the identified need to revise current behaviour change approaches to consider varying needs, outcomes, and cultural contexts.

The research applied the conceptual lens of context and complexity, which acknowledges that projects don't always follow the planned theory of change and that project operate in complex contexts that shift over time. "Activities may be implemented differently than planned, and individual stakeholders - including community members and implementers - will understand and respond to project activities differently based on their knowledge and prior experiences. Understanding how projects operate in such complex environments, then, is critical to know how SBC projects affect communities and households (and vice versa) and ultimately, how projects achieve or do not achieve expected and sustained outcomes."

The research, which was carried out in two phases and included a desk review, primary data collection and secondary data analysis, and key informant interviews, focused on three main assessment areas:  
 

1. A secondary data analysis, which sought to assess the effect and impact of SBC interventions supported by USAID and partners over the past seven years on health literacy, attitudes, social and gender norms, and healthy practices in different subpopulations. Subpopulations included those defined by geographical locations, age, education, socio-economic status, rural-urban residence, and proxies for cultural contexts. In addition, the assessment reviewed the data collection instruments and indicators to look at how these can be improved.
2. SBC assessment, which sought to identify best global practices in several areas of SBC and to assess how implementing partners in Ethiopia documented processes and procedures aligned with those practices.
3. Costing and cost-effectiveness assessment, which involved reviewing data and literature relating to SBC costing, examining SBC expenditures in Ethiopia, and applying modelling tools that assist in cost-effectiveness analysis.

The report shares findings for each of these assessment areas. The following is a brief synopsis of some of the findings, as highlighted in the executive summary:   

Lack of documentation: Agency for All was often disappointed to find that adequate documentation was unavailable, which was a key limitation of the assessment. In many cases, the documentation made available by implementing partners was typically in the form of quarterly or annual reports. These reports naturally highlight quantitative progress on intermediate results and do little to show how basic processes of materials development, implementation, and monitoring and evaluation (M&E) were undertaken.

A new KAP-gap: The assessment highlighted a "new KAP-gap", one in which implementer documentation and interviews consistently demonstrate technical knowledge and positive attitudes toward best practices in their field(s). However, such Knowledge and Attitudes frequently fail to translate into Practice. A question that came up frequently throughout this assessment was, "How can implementing partners be incentivized or persuaded to observe best-practices of which they are clearly aware but which they do not seem to follow?"

Belief in adaptation vs. belief that fidelity ensures outcomes: The assessment shows that implementing partners are being torn between, on the one hand, embracing the need to adapt programmes to local circumstances and shifting local realities, while at the same time being wary of too much adaptation due to the risk of evolving away from the objectives of that intervention or diminishing its effectiveness, often showing an uncritical belief that being faithful to generic protocols leads to programmatic success.

The importance of creating a foundation for cost-effectiveness analysis: The assessment found limited evidence on the costs and cost-effectiveness of SBC interventions in Ethiopia and argues for the importance of collecting costing data from implementing partners. Without collecting such data as part of a broader "costing framework" at the beginning of any intervention, programmes will be unable to meaningfully conduct the cost-benefit analyses critical to determining cost-effectiveness.

The fact that information requires careful targeting and reinforcement: The assessment suggests that the effects of some programmes on specific sub-populations can be low and non-significant despite overall significant improvements, as the impact of the interventions unsurprisingly varied across regions, education levels, gender, and age groups. Researchers believe that a root issue is developing targeted strategies that focus on specific populations. This was the focus of the THDR programme but should be a focus for all SBC.

The community's seat at the table: Best practices in community engagement were often not focused on community "engagement" but on ways that project personnel could "mobilise" the community by leveraging community leaders and other influencers as information distributors. Maintaining a robust and sustained interface with the community through interpersonal communication needs to become a central priority to ensure the quality of community engagement.

While the annexes in the full report provide a detailed list of recommendations for all three assessment areas, the following is a brief summary:   

Monitoring, evaluation, and learning:
 

• Design M&E efforts to provide information on which elements of the programme work for whom and under what circumstances.
• Encourage indicators to assess the factors hypothesised to influence desired programme outcomes and the outcomes themselves.
• Consider the complexity of behaviour when creating proximal indicators, including indicators that are easily adaptable to changing contexts.
• Monitor improvement comprehensively and holistically, moving beyond targets met.
• Align M&E indicators to the programme's theory of change (TOC), and be sensitive to changes at both the population and subpopulation levels (e.g., community, household, and individual levels). 
   

Social and behaviour change:
In total, there are 19 recommendations. The following items are just a selection:
 

• Because the fourth Ethiopian Health Extension Program (HEP) Goal is to "strengthen community engagement and empowerment", there should be a clear community engagement strategy that is conceptually distinct from community mobilisation. 
• Documenting community input should be improved and made accessible to all project partners.
• The idea of a TOC is critical and has been addressed by some of the implementing partners. However, where used, it is not clear that these national and regional TOCs were used to guide activity selection and implementation.
• All SBC projects that are "integrated" into broader programmes (e.g., service delivery) should develop an explicit (though brief) theory and "strategy for integration" that recognises the fundamental role of behaviour.
• A risk assessment and mitigation plan should be developed as part of a stakeholder analysis at the formative stage of project design.
• Most SBC activities are simply information distribution (via print, meetings, radio, or mobile narrowcasting, for example) without complementary efforts to help communities reflect and apply new information to their own circumstances. Opportunities for reflection and application should be created.

Costing and effectiveness:
 

• SBC costing and cost-effectiveness studies in Ethiopia should focus on key questions that the Mission and other stakeholders would like answered to best guide future SBC programming.
• When planning an impact evaluation, consider whether adding a costing component is feasible and whether the results will address the primary research questions for SBC costs and cost-effectiveness.
• During the contracting phase for programmes that include SBC, the contract should specify that reporting on SBC expenditures is required. Organisations bidding on programmes with an SBC component should report how their accounting and financial systems will track SBC expenditures, and reviewers should ensure that these efforts will be sufficient to answer key cost-effectiveness questions.

Click here for the 12-page executive summary in PDF format.