Hello, AEA community! We are Kate Krueger and Norma Ruiz from the International Republican Institute, a nonprofit, nonpartisan organization that supports democracy worldwide. In this year of upheaval, all evaluators — not just those of us who work in the DRG (democracy, human rights, and governance) sector — are witnessing the power of social movements and civic networks to challenge norms in even the most entrenched social and political institutions. One thing we do as DRG evaluators is assess the effectiveness of civic networks. This is challenging, as their boundaries can be hard to define and their goals evolve over time.
Our Washington DC and Mexico City offices worked together to develop a suite of MEL resources with support from the US Department of State’s Bureau of Democracy, Human Rights and Labor. This “Networks Field Guide” was built to assess civic networks in complex contexts. We have now used this guidance to inform IRI programming in a variety of operating environments. Here are a few tips that might assist your own network evaluation efforts.
Hot Tip: Balance precision with flexibility by “gamifying” design and MEL decisions for program team use.
Our first challenge was figuring out how to define the term “network” in our own organizational context. (This is a word that funders, implementers and evaluators alike often use with varying conceptual precision – check out this rad resource for more information). Networks are living creatures that both help shape and adapt to their context. We found that trying to create typologies to inform programming approaches resulted in frustration: programs actually preferred to see networks as a continuum rather than a set of discrete categories. So, that’s what we did! We created a series of simple quizzes and questions for program teams to answer to determine where their projects fell on a simple continuum of network designs. This interactive process built program team buy-in for subsequent MEL approaches.
Hot Tip: Take a “choose your own adventure” approach to indicator selection. Use indicator options to instruct!
After we got teams comfortable enough to define their networks with more precision, we provided a set of MEL tools that were equally as flexible, yet distinct. Using a simple 2×2 rubric of basic network characteristics such as structure (e.g. must your network have a clear hierarchy to achieve its objective?) or engagement (e.g. must your network have frequent activities to achieve its objective?) we provided a list of characteristic-based indicators, including qualitative lines of inquiry and sample evaluation questions, for program teams to use. This mix-and-match model helps program teams choose the indicators that truly matter to the type of results they want their network to achieve, while also providing guidance for why particular concepts, such as network incentive structures and membership composition, actually matter. A well-facilitated indicator selection process can be a powerful program design tool.
The American Evaluation Association is celebrating DemTIG Week with our colleagues in the Democracy and Governance TIG. All of the blog contributions this week come from our DemTIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
When do you introduce participatory methods to evaluate your networks’ activities?
I’m researching the use of participatory methods in evaluation and would welcome studies you’ve done/know of that use participatory methods in evaluation… not just in design.
thanks,
Janet
jkerley_22041@yahoo.com