Evaluating Social Justice Advocacy: A Values Based Approach

"Evaluations sometimes focus only on donor needs or efficient processes without taking into account how the evaluation process can model social justice concepts."

From the Center for Evaluation Innovation, this brief offers ideas on how to incorporate the concept of social justice and its underlying values into advocacy evaluation. It points to ways in which social justice values should influence what evaluators examine in terms of advocacy goals, theories of change, outcomes, and strategies. It also considers how the evaluation process itself can promote social justice values.

Author Barbara Klugman begins by exploring the meaning of social justice advocacy, which she describes as working for structural and enduring changes that increase the power of those who are most disadvantaged politically, economically, and socially due to their race, ethnicity, economic status, nationality, gender, gender expression, age, sexual orientation, and/or religion [LaMarche. G. (2009). Social Justice: A Guiding Vision for Atlantic's Final Chapter]. Klugman then examines 3 social justice values [Fraser, N. (1997). Justice Interruptus: Critical Reflections on the "postsocialist" Condition. New York: Routledge; Fraser, N. (2006), Social Justice in the Knowledge Society: Redistribution, Recognition, and Participation]:

1. Distributing resources equitably;
2. Recognising people in all of their diversity; and
3. Ensuring participation, which involves creating mechanisms for building public participation in policymaking, implementation, and monitoring as part of efforts to hold democratic governments accountable.

As Klugman explains, values-based evaluation has 2 meanings; the first is using evaluation to judge how well values are integrated into practice. This entails asking questions such as: To what extent did those with the least power play a part in shaping the goals? If so, to what extent did those people know about the political, economic, and social forces shaping their situation, and the short and long-term consequences for their lives? She states that considerations should focus on the dynamics that went into identifying a particular problem as the focus of advocacy, the process for identifying proposals to address this problem, and the process for gaining public, policymaker, and administrator interest in those proposals. In addition, she suggests that social justice advocates should consider the interim outcomes they need to achieve in order to meet their goals. The outcome categories she identifies here are [The first 3 outcome categories come from Reisman, J., Gienapp, A. and Stachowiak, S. (2007). A Guide to Measuring Advocacy and Policy. Seattle, WA: Organizational Research Services (The author added the fourth category.)]:

• Strengthened organisational capacity, which involves, in part, examining the extent to which organisational management, staffing, and involvement of grantees promote equity and value participation. This includes: the degree to which organisational members are able to contribute to an advocacy effort's direction; whether leaders engage staff and listen to their perspectives; and whether new leaders are developed.
• Strengthened base of support, which involves asking: Are problems defined and proposals generated from the perspectives of those who are most affected? Klugman here discusses the frequent need for - and challenges associated with - drawing in outside expertise yet still fully valuing the insights and experiences of those who are most affected.
• Strengthened alliances, which involves assessing whether the challenges of meaningful participation have been addressed when the state or other entities such as corporations, the media, or religious institutions are brought into the picture. Here, "ongoing assessment of decision-making transparency is particularly necessary..."
• Increased data and analysis from a social justice perspective, which involves exploring research questions to be sure they do not reflect the concerns of a particular interest group, as well as investigating judgments about what constitutes valid evidence, as these tend to be shaped by particular paradigms.

Klugman stresses that policy outcomes themselves should also be examined against the 3 social justice values described above. Questions evaluators can ask include: Does the policy specify how the public will be informed and engaged during implementation? Does it enable those on the margins to participate in monitoring activities? Does it take into account the needs of specific groups, such as blind, deaf, or illiterate people, when it comes to these communications?

She then explores impact: part of the long-term vision. This includes assessing shifts in social norms (e.g., decreased discrimination against a specific group or an increase in beliefs that the state should provide services for indigent people) and shifts in population-level impact indicators (e.g., lower school dropout rates for transgender youth because of declines in bullying).

There are also questions to be asked about the progress of the advocacy strategies. For example: "If the strategy involves public dialogue, for example, questions exist about how the problem and proposals to address it are articulated. What perspectives will attract the desired media attention, and does this approach fit with social justice values? How were decisions made about the public face of a campaign, and does this process and the choice of public speakers adhere to social justice values? If the strategy involves public protests, were protestors involved in identifying this strategy? Do they understand the implications both personally (in terms of personal time, costs, and security) and for their community (where backlash might undermine community welfare through curfews or shop closings, for example)?"

In the next section, Klugman explores the second meaning of values-based evaluation: incorporating social justice values into the evaluation process itself. The topics she covers here include incorporating social justice values when planning and undertaking an evaluation in: relationships with advocates, in theory of change development, in evaluation reporting, and in capacity building. For example, "[t]he evaluation process and the evaluator need to be respectful, recognizing the capacities and constraints of those whose work is being evaluated, aiming to enhance their capacities, and minimizing their reporting burdens where possible."

Finally, she looks ahead, with the hope that "the broader advocacy evaluation field will make values-based approaches a regular part of discussions about how best to meet the challenges of evaluating advocacy."