My name is Bob Williams. I’ve been involved in the systems field since the mid-1970s. Over the past forty years I’ve learned to distinguish between three different approaches to systemic practice:
- Observing systems
- Thinking systemically
- Being systemic
These differences have major implications for evaluation.
First, the distinction between observing systems and thinking systemically. In an exceptional paper, Barbara Schmidt-Abbey, Martin Reynolds and Ray Ison – systems practitioners who know a lot about evaluation – identify two ways in which evaluation has applied systems ideas. They call them first-order systems evaluation and second-order systems evaluation. First-order systems evaluation is essentially a belief that there is a system out there that we can identify, can agree on, behaves according to certain rules and that can be meddled with. It is primarily about observing systems. They argue that first-order systems evaluation promotes variations on the status quo … potentially doing the wrong things better. In contrast, second-order systemic evaluation essentially treats systems as conceptual tools by which ‘reality’ can be interrogated and challenged without being completely constrained by that reality. It is a way of thinking not a way of observing. The paper states that some evaluation approaches (e.g., Developmental Evaluation and Blue Marble) aspire to and in some cases get quite close to second-order systems evaluation.
But I have found you need to be careful shifting from observing systems to thinking systemically. You can’t observe everything, you cannot think about everything, so how do you decide where to place the boundaries of your evaluation? Gerald Midgley is another leader in the systems field who has extensive experience in evaluation. In a recent webinar, he stated that the systems field has failed to promote successfully one of its core principles. Thinking systemically is not about thinking about everything. Every systemic inquiry is partial. Far from being comprehensive, Gerald stated that systems thinking helps us deal with the inevitable lack of comprehensiveness of any practice. Holism is not about including everything, it’s being very thoughtful and informed about what you leave out. This idea about systems inquiry including everything likely came from the management field not the systems field. Indeed, when I hear evaluators refer to ‘the systems level’ or even ‘the big picture’, generally they are thinking managerially and not thinking systemically.
Finally, being systemic. Historically, evaluation has been driven by method; proposals frequently prioritise method over objective. Michael Patton has called this ‘obsessive method disorder’. Being systemic moves beyond method, ‘thinking’ or ‘observing’. It is about understanding the core elements of systems approaches and incorporating them into whatever you do. It is about treating boundary choices as purpose and values-driven rather than value-neutral decisions. In that sense being systemic is not unlike the kind of principle-based evaluator envisaged by Principle Focused Evaluation.
The question is what kind of systemic evaluator are you? Are you an observer, a thinker or a be-er? Are you all three or just one or two? What kind would you like to be?
Rad Resources
Here are some resources that will help you.
- Systemic evaluation design. Bob Williams. A free workbook and a 20-minute video.
- E Book – https://gum.co/evaldesign
- Video –https://www.youtube.com/watch?v=VUTqG5j21mU
- More on my website http://www.bobwilliams.co.nz
- Towards systemic evaluation in turbulent times – Second-order practice shift. Barbara Schmidt-Abbey, Martin Reynolds & Ray Ison (Free Access).
- Systems Approaches for Accelerating the SDGs and Building Back Better. Gerald Midgley’s presentation starts at 25.00 ends 54.00
- Principles Focused Evaluation
The American Evaluation Association is hosting Systems- and Complexity-informed Evaluation Week. The contributions to AEA365 this week are all related to this theme. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.