AEA365 | A Tip-a-Day by and for Evaluators

Mar/13

22

ECLIPS and Systems TIG Week: Tarek Azzam and Matt Keene Recap Systems-Oriented Evaluation

Hello! We are Tarek Azzam (Claremont Graduate University) and Matt Keene (Environmental Protection Agency). We are members of the External Review Panel for ECLIPS.

Whoa! Even though it’s oh-so-tempting to try, you don’t need the perfect string of words to define “systems-oriented evaluation.”

Lesson Learned: It’s in the roots of evaluation 

At its core a systems approach to evaluation encourages the evaluator to consider the physical, political, and structural issues that surround a program, and to examine how these factors help or hinder the success of a program. This examination and reflection process is incorporated in the work and writings of lead evaluation scholars such as Lee Cronbach, Robert Stake, and Jennifer Greene. The presence of systems thinking also can be seen in our standards (specifically standards F3, A4, and A7).

And it’s also something different, because it requires the evaluator to recognize that the program is part of systems that have their own dynamics. It forces the evaluation to examine issues that go beyond the process and outcomes of a single program.

Lesson Learned: How to become a systems-oriented evaluator

1)      Adopt some habits of systems thinkers

ECLIPShabits

2)     Know the domains of Social Ecology and use them to understand the leverage points of boundaries, relationships, and perspectives. Donella Meadows says that boundaries are problem dependent and messy. Don’t make the world linear for your mathematical or administrative convenience.

ECLIPSdomains

3)     Delve into the dynamics of systems to find the regions of organized, adaptive, and unorganized patterns.

ECLIPSadaptive

4)     Find leverage points (places to intervene where small tweaks can lead to big changes). Here are some leverage points from Meadows you can use to influence relevant systems:]

ECLIPSpoints

5)     Let systems thinking do fuzzy things to your logic model. A fuzzy logic model takes into account the dynamic nature of the systems surrounding a program. It gives a visual image of the complexities that can affect processes and outcomes.

In the ECLIPS, all of our logic models used to look like this:

ECLIPSfuzzy

But after applying systems thinking, we made them into fuzzy logic models. See how different they look.

ECLIPSexamples

Try creating a fuzzy logic model to find and depict the system’s complexity, making your logic model more useful to people for a longer time.

That’s all for now

This concludes ECLIPS week on aea365! Don’t expect to learn about systems alone or in a short period of time. It may well be a journey into a new way of thinking about evaluation. Get involved in an existing community of practice about systems or form your own group. ECLIPS members are happy to share what they are doing.

The American Evaluation Association is celebrating this week with our colleagues involved in ECLIPS—Evaluation Communities of Learning, Inquiry, and Practice about Systems—and the AEA Systems in Evaluation Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

3 comments

  • Mohammad Hassan Mohaqeq Moein · September 18, 2013 at 1:54 am

    Salaam , Nice piece! Mamnoon

    Reply

  • Kylie Hutchinson · March 29, 2013 at 4:58 am

    This has been a great week on system-thinking and evaluation. It was a perfect introduction of basic principles and has thoroughly whetted my appetite. The timing was also great for something I’m working on. Thanks!

    Reply

  • Chad Green · March 25, 2013 at 5:30 pm

    The systems thinking habits have always intrigued me. What framework was used to create them?

    Thanks,
    Chad

    Reply

Leave a Reply

<<

>>

Archives

To top