Feedback Mechanisms in the Former Yugoslav Republic of Macedonia

"[R]eviews of recent humanitarian responses - to both natural disasters and complex contexts - often find that mechanisms to ensure accountability to the affected community are infrequently embedded into the response, and if they are, it is very rare that a response is altered based on the feedback collected."

This case study draws recommendations from United Nations High Commissioner for Refugees (UNHCR)'s operation in the Former Yugoslav Republic of Macedonia (fYR Macedonia) that demonstrate how effective feedback mechanisms can be used as a tool to inform and shape humanitarian response. The establishment of the inter-agency feedback mechanism in fYR Macedonia, the Children and Family Protection Support Hubs (CFPSH), which are described in the case study, was supported by UNHCR Innovation's Emergency Lab, which has a specific focus on communicating with communities.

The CFPSH initiative was an attempt to address the challenges UNHCR faced in light multiple conflicts in the Middle East and Afghanistan - catalysing the movement of thousands of people seeking protection across Europe. UNHCR identified the need to enhance the way it engaged with refugees and migrants to make it more comprehensive and consistent. One of the elements of the CFPSH hubs was Information and Advice Desks (IADs) - or information kiosks - which were established in transit centres at the northern and southern borders (in Tabanovce and Gevgelija respectively) as visible, static information points. The IADs serve as the central point for feedback and information requests and allow for feedback across all services and issues to be captured in each location. An image featuring a blue dot represented the CFPSH to make it easy for families to seek out the IADs across different sites, both inside and outside fYR Macedonia. Standard Operating Procedures (SOPs) were created with a mobile-data collection function introduced so that staff could collect feedback in real-time.

As detailed here, both the United Nations Children's Fund (UNICEF) and UNHCR co-hired the staff through the local non-governmental organisation (NGO) LaStrada and provided specific training. UNICEF provided the equipment for the desks, and UNHCR took on the responsibility for providing the technical information management solution for capturing and storing feedback data. This included the overall responsibility for analysing and sharing the data with the appropriate organizations in the available forums. An online mechanism called Kobo Toolbox was designed to support data collection, visualisation, and analysis. In fYR Macedonia, these IADs were personned by a dedicated team of assistants who also had a roving function - walking the sites to improve outreach and information provision. UNHCR with UNICEF and local partners trained the IAD assistants in engaging with people in a culturally appropriate way and capturing the relevant details of their interactions. This role also included management of individual cases and ensuring follow-up. Effort was made to ensure that staff are easily identifiable and accessible so that people knew they could be approached for information and advice and as a channel to provide feedback. Many of these assistants also speak Arabic, further helping them engage directly with people.

In conjunction with UNHCR Innovation's Emergency Lab, the fYR Macedonia partners conducted an information and communications needs assessment with refugees and migrants to identify their preferred modes of communication. An outcome of these conversations was that face-to-face communication is often preferred, so the SOPs were designed by mapping different pathways according to how individuals would respond to somebody coming to the desk in person. Forms were created based on the information that was needed - it was important not to "over collect" information that would be irrelevant or superfluous. Whilst this mechanism is centred primarily around the information kiosks and its assistants, there are several aspects that make the fYR Macedonia mechanism more plural in terms of communication channels. For example, a phone number was advertised in women's bathrooms for sensitive complaints.

Based on the Emergency Lab's experience in a number of humanitarian contexts and specifically drawing on learning from the inter-agency feedback system in fYR Macedonia, the case study outlines 10 key steps to setting up an effective mechanism. As summarised by Katie Drew and John Warnes of the UNHCR Innovation Service, the steps include:

1. Define the Challenge: Consult with communities and humanitarian responders to work out what barriers are preventing feedback being listened to and acted on. This homework stage is essential. Work out where the problems really lie - these can include lack of staffing, poor or non-existent communication channels, non-working technology, bad data management, and confused information flows.
2. Don't duplicate efforts: Build on existing staff capacities and work with established services to ensure sustainability. Work out what activities are already underway and build on these, identifying which community members, partners, and staff you can work with.
3. Use the community's preferred communication channels: Engage with them through channels they like and trust. Conduct an Information and Communication Needs Assessment - different groups and individuals prefer different channels.
4. Coordinate: Have clear commitment and agreement on roles and responsibilities, both internally and inter-agency. Draft SOPs that clearly indicate the agreed commitments, timeframe for follow-up, and different agency/individual roles.
5. Focus on what you need to know to make improvements: Design your mechanism to collect structured data that enables you to make decisions and take action at the right time. Mobile data collection enables quick collation, is time-efficient, and reduces errors. It also enables the capture of standardised data based through structured questionnaires. Design the questionnaire with decision makers so they define what is information is collected.
6. Prepare for sensitive issues: Ensure you have the resources and appropriate channels to create a safe and confidential space for reporting. Provide private areas and/or channels for discussion and reiterate the confidentiality of the information being shared. Always consider gender and age dynamics of staffing and ensure they know how to escalate/refer the sensitive issues they receive.
7. Test and refine: Speak to those using the mechanism and iterate your design to ensure effectiveness. Consult with communities to understand why/why not they are using the mechanism.
8. Enable evolution: Adapt the mechanism to the changing dynamics of the response – go back to the drawing board if necessary. The more channels that are established, the less risk there is of complete loss of engagement with a community if access to one fails.
9. Make data digestible: Visualise the data, show trends, and find the right forum(s) for sharing to make it accessible. Mobile data collection tools help easily pull data into charts – show trends for different groups and clearly highlight priority concerns for follow-up. (In fYR Macedonia, using Kobo Toolbox, a focal point creates weekly reports that are distilled to highlight key issues requiring action. These reports also include an overview of all information received, including positive feedback. Sensitive, individual case data are not shared. The mechanism subsequently adopted a supplementary monthly "visualisation" of the data to clearly demonstrate changes in perceptions and trends. On an intermittent basis, broader trend reports are created with greater focus on prioritisation of areas of focus, resolving longstanding issues and steering the overarching direction of the humanitarian response. The feedback from decision makers on this reports indicated that they were accessible and easy to navigate.)
10. Demonstrate you have listened: Proactively explain the changes you have made and why certain actions sometimes cannot be taken. Don't wait until the community is frustrated by the lack of feedback make sure a key responsibility of the mechanism is to "close the loop" through preferred and trusted channels.

Prior to concluding sections that provide practical advice for setting up mobile data collection, the case study includes reflections on the UNHCR operation in fYR Macedonia to date. The Kobo Toolbox reports "led to immediate action from field times on the relevant issues to ensure timely follow-up and improvement of humanitarian delivery, based on the needs....Over time, the team in the Former Yugoslav Republic of Macedonia will further improve the way it captures and analyses the data it is pulling in through the information and advice desks. Efforts are being made to explore whether additional channels can be established to bring in feedback from the affected communities."