Hello, Keiko Kuji-Shikatani (C.E., CES representative for EvalGender+) and Hur Hassnain (Pakistan Evaluation Association; Impact, Results and Learning Manager, Y Care International) here to share our thoughts on how to engage and collectively think about better evaluating learning and social accountability in FCV (fragility, conflict and violence).
The World Bank estimates that by 2030, the share of global poor living in FCV is projected to reach 46%. According to the OECD, ‘fragile states’ are most at risk of not achieving the sustainable development goals.
Hot Tips and Rad Resources:
Here are seven Hot Tips and Rad Resources to consider wh
en evaluating in FCV:
1-Context. Take context as a starting point and invest in FCV analysis to understand sources of tension and cohesion.
2-Be conflict-sensitive, whilst working in FCV we need to realise that no one is neutral. Evaluations should explain the interactions between context and the intervention.
3-Good monitoring precedes good evaluations. Traditional periodic evaluations are unrealistic when evaluators struggle to have access to the targeted people. Monitoring supports adaptive programming by informing decision makers faster, resulting in timely project fixes.
4-Engaging local communities where access is restricted, in the M&E processes to make them agents of change. This requires a well-planned and thoughtful process to ensure their safe and meaningful involvement.
5-Third Party Monitoring. TPM is a risk-management to
ol intended to provide evidence in inaccessible areas, it also presents some ethical and technical limitations. The Secure Access in Volatile Environments program suggests TPM works best when used as a last resort.
6-Using information and communication technologies where remote programming is needed, ICTs offer creative solutions to compensate face-to-face interaction, making evaluations an agile tool for adaptive-management; new ethical challenges and the new kinds of risks that digital data brings need to be mitigated. See Oxfam’s Mobile Survey Toolkit for tools and providers.
7-Is the evaluation worth the cost when money could otherwise be used to relieve human suffering? Think twice if the context is fluid, continuously changing and the target population is on the move. Cost is justified only if the findings have the capability and potential to lead to program improvements andgenerate learning without compromising the security of the affected population, people delivering aid or collecting data. Depending on the context you can choose from a spectrum of options including more informal reflective learning exercises (e.g., After Action Reviews/Real-Time Evaluations) and use user-friendly communications including social media posts with the evaluation participants.
A greater drive for meaningful conflict-sensitive evaluations that investigates the causes of FCV, instead of ‘fig leaf’, evaluations would contribute to better outcomes and new policies to provide more flexible and faster support for those whose lives are torn apart by war and conflict.
Interested in learning more? Reach out to the International Development Evaluation Association who with its partners established a Thematic Interest Group on Evaluation in fragility, conflict and violence (EvalFCV).
The American Evaluation Association is celebrating International and Cross-Cultural (ICCE) TIG Week with our colleagues in the International and Cross-Cultural Topical Interest Group. The contributions all this week to aea365 come from our ICCE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Pingback: Seven Tips to better evaluations in fragility, conflict and violence – Pakistan Evaluation
Thank you Samandar for your comment and sharing your learning from Kandahar. ICTs could indeed be a great tool where we are required to reduce human contacts. Kandahar is indeed a very volatile context.
I have one question: was it enumerators inputting the data or respondents themselves? Also did you feel farmers of Kandahar were comfortable with sharing their data on a mobile device?
In one of my recent studies in Pakistan respondents refused to give information on a mobile/tablet because of their fears with these technologies.
Great blog and well aligned to the current contexts of Fragility, Conflicts and Violence. I have one comment or additional info for the tip 6 based on a recent experience.
During December 2016 and March 2017 I led a Survey in Kandahar Province of Afghanistan, which is relatively an insecure province, specially the rural areas are more conflict affected, and insecure. In terms of data collection, it is impossible for people outside of the province to go and collect data for any survey, (this related to tip 4). The following ICT technologies were used in this survey, which was a great learning.
1. Open Data Kit (ODK), which is a data collection software was used in smartphones to collect responses, and the ODK software embedded the GPS locations through internet to ensure that the questionnaire was filled at the sampled geographic locations.
2. WhatsUp was used to allow enumerators contact the survey manager from the field for any questions or guidance, and for sending update of daily interviews to supervisors. A chat group of all surveyors was created in Whatsup.
The survey was with farmers to assess a project supporting farmers to learn new technologies in harvesting and post harvesting farming practices to increase yields.
I hope it is helpful.
Thanks Samandar for your comment. Very interesting insights from Kandahar, Afghanistan. Certainly one of the worst conflict affected area in the country. I am interested to know if people were fine to respond to tablets/mobile phones there? In one of our recent evaluations in rural Pakistan, we found that people especially women were not comfortable with using technology due to lack of trust and many other reasons. I would love to hear about your experience in this regard.
Dear Hur, I missed your questions and just returned to the blog now and saw you posed questions about my comment.
Re your question whether the data was recorded by enumerators or the respondents, and whether the respondents were comfortable sharing their data on a mobile device.
Answer: The data was recorded in tablets by the enumerators, and the farmers (respondents) were comfortable with the use of technology (mobile devices / tablets) to share their answers.
However, it would have been difficult to get responses from the farmers in such a volatile context, the reasons they were comfortable includes the followings:
1. The farmers who responded to the survey were program beneficiaries, and they had a trust established with the organization and the project.
2. The enumerators were the social capital officers of the project, who had significant interactions with the communities, farmers and different community-based organizations, and they were well respected among the community, so they were more trusted.
3. The project was very successful and had tangible outputs delivered to the communities, such as irrigation canals were rehabilitated, retaining walls against floods, and vineyards were rehabilitated by the project.
4. Local communities were not only engaged in the survey, but also in the project design, identification of interventions, and appraisals. The project was using a tool/approach called CBPSA (Community-Based Planning to Support Alternatives) which engaged local communities in all aspects of project design, to delivery of services; so the communities in which the survey was conducted already had enough information about the project and the role they had played.
Once again, thank you for the great blog.