Bridging the Gap: The Role of Monitoring and Evaluation in Evidence-Based Policy Making

This 220-page publication explores the strategic role of monitoring and evaluation (M&E) in evidence-based policy making. Its contributors are senior officers in institutions including national and local governments, the United Nations Children's Fund (UNICEF), the World Bank (WB), and the International Development Evaluation Association (IDEA), and it was published in collaboration with WB, the UN Economic Commission for Europe, IDEA, the International Organization for Cooperation in Evaluation (IOCE), DevInfo, and Multiple Indicator Cluster Surveys (MICS). These and other members of the international community reportedly agree that M&E has a strategic role to play in improving the relevance, efficiency, and effectiveness of policy reforms. A core proposal here is that the dialogue between the suppliers and users of evidence should be strengthened to bridge the gap between the information needs of policy-makers and the information offered by researchers and evaluators. This publication is one contribution to that conversation.

According to the book's editor Marco Segone, the concept of 'evidence-based policy making' has been gaining currency over recent years. He understands evidence for policy as having 3 components:

1. hard data (research, evaluations, etc);
2. the analytical argumentation that puts the hard data into a wider context; and
   
3. an evidence base comprising stakeholder opinion.
   

The first few contributors explore evidence-based policy making through a broader scope, noting - for example - that evaluation is always couched within a political context. Relationships must be shaped in recognition of this reality. Another chapter places M&E in the wider context of knowledge management, as an element of organisational learning and performance strengthening. It focuses on the case of UNICEF, using the example of a monitoring system linked to research and policy development in the region of Central and Eastern Europe (CEE) and the Commonwealth of Independent States (CIS). The next 2 collaborators explore how to design and implement a results-based M&E system designed to assess whether and how policy goals are being achieved over time. The presented "Ten Steps Model" addresses the challenge of how governments in general, and those in developing and transition countries in particular, can create systems that provide credible and trustworthy information - both for their own use and for the benefit of their citizens.

The following few chapters focus on the strategic intent of evaluations, studies, and researches. One core question explored here is why the utilisation of evaluation findings is disappointingly low, despite the significant resources devoted to evaluation and its growing importance in industrialised, transition, and developing countries. Segone himself argues (in an earlier essay within the book) that attention is more likely to be paid to research findings when:

• "The research and evaluation is timely, the evidence is clear and relevant, and the methodology is relatively uncontested.
• The results support existing ideologies, are convenient and uncontentious to the powerful.
• Policy makers believe in evidence as an important counterbalance to expert opinion, and act accordingly.
• The research and evaluation findings have strong advocates.
• Research and evaluation users are partners in the generation of evidence.
• The results are robust in implementation.
• Implementation is reversible if need be."

Several chapters focus on country-led evaluation, which means that "[c]onventional forms of evaluation, typically mandated and funded by development agencies, are now being challenged by emerging independent forms of assessment which put the recipient country in the driver's seat." For example, one contribution presents the experience of a joint Country-led evaluation of the policies related to child well-being within the social protection sector in Bosnia and Herzegovina. The authors of this piece explain how it was possible to accommodate the information gap of both the Government and UNICEF, while ensuring an independent and objective evaluation process.

The final sections of the publication focus on data collection and dissemination as elements crucial to both policy making and the evaluation function. One UNICEF-supported international household survey initiative - the Multiple Indicators Cluster Survey (MICS) - is designed to assist countries in filling data gaps for monitoring human development in general and the situation of children and women in particular. As Segone puts it, this survey "has been instrumental in strengthening national statistics capacities, highlighting and filling gaps in quality data, monitoring and tracking progress toward national and international development goals and, in identifying emerging development issues and disparities among groups in societies."

However, he notes, collecting quality data is necessary but not sufficient. Data must also be disseminated in a user-friendly way to ensure that they are understood and used, and therefore inform policy decisions. One contributor explains how a database system, DevInfo, has harnessed the power of advanced information technology to compile and disseminate data on human development. The system, which is endorsed by the UN Development Group to assist countries in monitoring achievement of the Millennium Development Goals (MDGs), has been adapted and used by more than 80 governments and donor agencies, and more than 10,000 professionals have been trained in its use.

Concrete examples are provided of how DevInfo has been used to strengthen integrated national M&E systems and to enhance advocacy and public awareness on priority development issues. For example, data presented through DevInfo are gradually playing a strategic role in facilitating a common understanding among Moldova's government, civil society organisations (CSOs), and development partners. Data analyses and maps are used as platforms for the national dialogue on poverty reduction. DevInfo is used to produce a bulletin on the national Poverty Reduction Strategy (PRS) implementation, which is published regularly in Moldovan newspapers and posted on government websites. According to Segone, this national process was instrumental in the government's decision to invest up to 21% more in the social sectors in 2006. In Serbia, Segone explains, the national DevInfo database contains a set of indicators which are used to monitor the MDGs, the PRS, and the National Plan of Action for Children. In addition, DevInfo is used to inform policy decisions at municipal level. In the municipality of Pirot, DevInfo and local situation analysis revealed that social services had overlooked many children being left out of education, in particularly Roma children and those with disabilities. Now a local team of Roma representatives and educational experts are working to prepare the ground for continuing education of Roma children. Also, informed by DevInfo data, "an increasing demand from the local population for a better quality of child social services prompted local authorities to provide additional funds. As a result, investment for children was increased 7 fold in just two years."