Hi – I’m Juan Carlos Peña, one of the Section Chiefs at IED.
I’ve found that evaluation culture in the UN is slowly, but steadfastly improving; however, there’s a lot to be done to ensure that everyone get on the evaluation train! From our unique oversight perspective, there are several critical and strategic challenges IED faces in conducting evaluations.
From big to small and everything in between, and with limited resources. One key challenge we face is evaluating a large and diverse set of entities and themes: from conflict prevention, peacemaking, and peacekeeping to promoting sustainable development. In any given year, we might cover a huge entity, such as the United Nations High Commissioner for Refugees, whose budget is over $7 billion, or the Office of Outer Space Affairs, with a budget of around $9.5 million. Regardless of entity size, IED is challenged to efficiently produce evaluations with teams, typically of no more than 2, sometimes 3, evaluators.
Evaluating while the train is moving. While this is not only a feature of the UN, the complex governance arrangements and dynamic operations often require IED to evaluate in the context of a very fluid and changing environment. These changing contexts influence the type of evaluations that IED might end up doing. IED has learned in the process the necessity of being able to conduct formative, summative, process, and outcome evaluations in parallel, and then trying to ensure that the entities improve and strengthen their delivery in the process.
What to evaluate and what data to use. Defining outcomes and obtaining reliable and robust data to assess performance is certainly another critical challenge. As evaluators we need to define “results” that are specific, tangible and measurable. Achieving world peace, eradicating hunger, achieving decent work and economic growth, reducing inequalities and ultimately achieving sustainable development are no small deeds. As such, a key lesson for IED is to calibrate how best to measure the work of the entities and to obtain enough evidence to determine their performance and contribution to greater outcomes and objectives. While a good point of departure are the entities’ own performance indicators, they are often output rather than outcome oriented.
Overall, I would say that the most critical challenge is finding the right pitch for our evaluations so that they are able to respond to the questions of whether entities are doing things right and are also doing the right things and at the necessary scale for them to have an impact. And in so doing, be able to inform Member States and managers on the efficiency, effectiveness and relevance of the UN.
The American Evaluation Association is celebrating Inspection and Evaluation Division (IED) of the UN’s Office of Internal Oversight Services. All posts this week are contributed by evaluators who work in IED. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.