Responding to Global Change: A Theory of Change Approach to Making Agricultural Research for Development Outcome-Based

"Theory of change represents our best understanding of how engagement and learning can enable change as well as how progress towards outcomes might be measured."

This paper describes a theory-of-change approach to an agricultural research for development (AR4D) programme, building on an understanding of how engagement and learning can enable change. The approach includes a monitoring, evaluation, and learning system that combines indicators of progress in research along with indicators of change aimed at understanding the factors that enable or inhibit the behavioural changes that can bring about development impacts. This approach is illustrated with reference to the CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS), a global partnership that unites organisations engaged in research and capacity development for a food-secure future. After providing some background on theory of change, the paper examines progress so far in implementing the approach in CCAFS, focusing on programme design and systems for planning and reporting. It concludes with a discussion of some of the lessons learnt regarding institutional change, monitoring and evaluation, and behavioural change.

AR4D is described here as a set of applied research approaches that aim to contribute directly to the achievement of international development targets such as the Sustainable Development Goals (SDGs). Research outputs contribute to behavioural changes, manifested in changes in knowledge, attitudes, skills, and practices of a wide set of non-research next users such as development practitioners, extension services, farmers, and policymakers. These outcomes in turn lead to impact, such as increased food security or reduced poverty. The aim of AR4D is not to take over the work of development agencies but to ensure that the outputs of research maintain their integrity and are appropriately contextualised (translated, communicated, and disseminated).

Theories of change can provide a means to make explicit the implicit, often elusive, hypotheses on the processes that bridge the gaps between research design, outputs, use, and outcomes (See Figure 1 in the full paper). Approaches based on theory of change have several implications for implementation in practice, such as the need for qualitative as well as quantitative monitoring of the performance of research projects, the need to involve a wide range of people in its design, and the need to regularly examine the assumptions associated with it and adjusting programme management accordingly.

As detailed here, the goal of CCAFS, which started in 2010, is to overcome the additional threats posed by a changing climate to achieving food security, enhancing livelihoods, and improving environmental management. Project implementation in CCAFS aspires to a "three-thirds principle" in relation to engagement effort: a third working with next-users to build relationships and define their needs from research, a third on the research itself, and a third on enhancing next-users' capacity to improve the uptake of research outputs. CCAFS's approach to theory of change is centred on adaptive management, regular communications between programme and projects, and facilitated learning within and between projects. Using theory of change approaches within AR4D requires the strengthening of scientists' capacities to do research differently, work across research disciplines, and work with non-research partners for impact, and that institutions facilitate such shifts. CCAFS implemented a one-week training course on using theory of change for project and programme planning; an online community of practice and wiki were established so that experiences could be documented and shared during the pilot phase. Besides periodic virtual meetings, project participants in the pilot phase were surveyed for a more in-depth and standardised reflection and for capturing lessons and achievements from their experiences. These lessons, from both a project and programmatic perspective, were documented in reports and a series of learning notes, available on the CCAFS website.

As is explained here, opportunities for changing the programmatic approach to project planning and implementation emerged towards the end of 2014. To test the approach, several CCAFS pilot project teams were tasked with shifting to a theory of change and learning-based approach for planning their projects. Impact pathways and theories of change were developed for all 4 thematic areas and for the 5 regional programmes in CCAFS as one step in refining the program portfolio. The iterative development of the theory of change and impact pathways for all 90 projects in the portfolio involved initial meetings that were held virtually; the process was completed in 5 regional face-to face meetings with key next-users and stakeholders actively participating. These regional workshops were of 3-4 days' duration each, with approximately 50-60 scientists and partners in each workshop. The workshops resulted in several outputs: impact pathways for many of the projects, key partners trained in theory of change development, and a set of outcome targets, as well as workshop documentation and learning notes.

Several examples of project theories of change are shown in Figure 2 of the full paper, a portion of which is displayed above. In Figure 2A in the paper, an approach is shown for bridging between science and policy in the climate and agriculture space at the global level. Activities in this theory of change emphasise partnerships, putting a large proportion of effort into understanding and responding to next-users' needs. A second illustration is shown in Figure 2B, at the national level. A set of regional scenarios was developed through a participatory process for Southeast Asia, and these were quantified with research partners and used in the formulation of the Climate Change Priorities Action Plan (CCPAP) of Cambodia's Ministry of Agriculture, Forestry and Fisheries. (The diagram, seen also above, indicates activities such as participatory development of quantified future scenarios for Southeast Asia at regional level through workshops and outputs such as media and blog posts.) "As for many AR4D activities, participatory development and engagement processes are key for attaining outcomes. Without stakeholder buy-in from the very beginning, research activity outputs may have no traction among partners and intended next users." A third illustration, at the sub-national level, is shown in Figure 2C, focusing on work in one state of India to improve weather-based index insurance in several states. As in the other examples, the theory of change for this work was co-developed early in the project cycle in an effort to ensure relevance and buy-in from all key stakeholders.

A monitoring, evaluation, learning, and impact assessment strategy was developed to support the new approach and to help promote an "evaluative culture" within CCAFS; it is described in the paper. One key component is an information and communication technology (ICT)-supported programme and project management process in the form of an online platform, building on the online community of practice mentioned above, used by project teams to plan activities, report on them, and monitor progress against outcome target indicators. This platform is accompanied by a "support pack" that provides practical guidance and tools for quantitative and qualitative monitoring. Project evaluation and synthesis are carried out in the wake of project reporting to simplify reporting to funding agencies and facilitate learning and knowledge brokerage across the programme portfolio and beyond.

The paper summarises the CCAFS experience in implementing an AR4D approach based on theory of change in relation to 4 overarching elements: flexibility, learning, effectiveness, and incentives. It is noted that an evaluation of the outcomes and impacts of CCAFS's work is planned for 2020-2021, building in part on revisiting the baselines carried out in the 5 target regions in 2011-2013. "In general, this kind of evidence generation will almost certainly involve multi- and trans-disciplinary research that mixes quantitative approaches to measure outcome variables with qualitative approaches that establish the causal mechanisms involved, however difficult this may be in relation to social processes and human behaviour....Above all, we need to avoid the focus on outcomes being seen as disadvantageous to science, and development being seen as in competition with the science. Rather, they need to be seen as complementary, enabling, and ultimately liberating." Ultimately, despite its challenges, outlined in the paper, the conclusion offered is that "[t]he shift to an AR4D approach based on theory of change is fostering real transition in the international agricultural research community, much of it for the better..."