EPE TIG Week: Evaluating Disruptors: Thoughts from Engagement with an Eco-village by Allison Van

Hello evaluators, my name is Allison Van and in addition to my day job as an internal evaluator at University of North Carolina, I do some consulting for community initiatives.   A year ago, I stumbled upon Hart’s Mill Ecovillage and Farm.

I was inspired by their mission of radically re-imagining relationships with the land and each other.  Soon I was gladly offering my skills in evaluation and organizational change to help them develop.  Little did I appreciate how much it would develop me.

Lessons Learned:

• Hart’s Mill is trying to build a radical alternative to conventional models – in just about every aspect of life. From collective ownership and sociocracy as a decision-making model, to decisions like what form of insulation to use in the homes, every decision is informed by the community’s environmental and social guiding principles.  This has compelled me to re-consider some of the assumptions that have crept into my evaluation practice over the last fifteen years.

Most fundamental among them is the notion that progress and stagnation are relatively visible and measurable with the right metrics.  I have long considered myself a developmental evaluator engaged with complex systems and programs with uncertain outcomes.  And yet here I realized the limits of my imagination about the level and degree of complexity possible.  Hart’s Mill is changing too fast and too fully to even consider a logic model.  Instead, being of assistance to them has been about documentation, reflecting back, and doing whatever will inform their next big decision.   It has also meant gracefully accepting that the next big decision will almost certainly change.

• The structure and approaches of classic evaluation and organizational change frameworks come with assumptions that I largely didn’t see or consider before. For instance, to prioritize efforts, I pulled out one of my favorite approaches, asking members of the community to vote with dots, where the most dots indicate the highest priority.  It didn’t work at all.  Some gentle feedback reminded me that the method assumes that the majority knows best, or that people’s initial opinions are informed enough for decision-making.
• I have built my career on the belief that quality information is critical to effective action.  I still deeply believe that.  And yet, I have come to realize that I implicitly ask others to prioritize data I provide over intuition, spiritual belief or other guiding forces in decision-making.  I have no simple answer to this, but I now have a deeper appreciation that I offer one means of understanding, not the only means.

In engaging with a community that seeks to disrupt the dominant culture, my own evaluative framework and culture have been upended.  As environmental problems lead to more radical re-imaginings of our relationships with the land and each other, I suspect that as a field, we will need to more fully explore what we invite and suppress in our work.

The American Evaluation Association is celebrating Environmental Program Evaluation TIG Week with our colleagues in the Environmental Program Evaluation Topical Interest Group. The contributions all this week to aea365 come from our EPE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.