Hi, I am Roman Katsnelson of KRD Consulting Group in Calgary, Canada. We provide organizational development supports to social good organizations across Canada and the US. As part of our work, we use evaluation practice and evaluation capacity building to amplify collective learning and augment collective efficacy.
Why does the work of neighbourhood development and revitalization continue to contribute to unintended outcomes, ranging from gentrification to NIMBYism? In a recent project surveying three major initiatives in Edmonton and Calgary, we found that when it comes to Community Economic Development, there is often a disconnect between proclaimed values and lived values. Despite best intentions, projects struggle to put into practice their stated commitment to community engagement and community development.
Lessons Learned: Community Economic Development projects should integrate a co-equal focus on impacts pertaining to all three domains: community (relationships and belonging), the economy (stabilization or growth), and development (structural change). On paper, all three of the projects we surveyed proclaimed such a commitment. In practice, two of them prioritized just economic growth, and the third just structural change – all three projects effectively deprioritized community outcomes.
A common trait of all the projects was their use of linear/predictive evaluation approaches that are less suitable to evaluating community change. Community impacts often get relegated to secondary status due to their inherent complexity and the difficulty in using traditional monitoring/evaluation techniques to effectively evaluate them. We’ve found the below set of complex-aware, systems-oriented approaches helpful in creating evaluation frameworks that do not abdicate the responsibility to make sense of community outcomes.
Rad Resource: The Cynefin Framework is a systems-thinking approach which classifies various components of an initiative (or various initiatives) by their relationship with complexity – is this bit of work “simple”, “complicated”, “complex”, or “chaotic”? Linear, predictive evaluation approaches are effective for simple and complicated domains; iterative and experimental approaches are more helpful for complexity or chaos. Check out Heather Britt’s primer on using Cynefin for evaluation.
Rad Resource: Critical System Heuristics (CSH) is a powerful tool that helps to shine the light on any gaps between our best ethical intentions and our practice, by asking a series of paired questions like “Who ought to be involved?”/“Who is involved?” and “Whose voice ought to count as expertise?”/“Whose voice is?”. CSH is a part of a systems-oriented evaluation toolbox.
Rad Resource: Outcome Harvesting (OH) is a highly flexible approach that allows for rigorous evaluation of both intended and unintended outcomes which have already happened, rather than measuring progress towards predetermined goals. Once outcomes have been surfaced (harvested), OH provides the interpretive structure to sort them into various themes – such as the three domains outlined above – to gain understanding of broader impact across all of them. Start here or get in touch with me to learn about Participatory Outcome Harvesting, which we are developing at KRD to integrate aspects of CSH directly into OH design.
The American Evaluation Association is celebrating Community Development TIG Week with our colleagues in the Community Development Topical Interest Group. The contributions all this week to aea365 come from our CD TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.