Hi, I’m Dawn Hanson Smart, Senior Associate at Clegg & Associates, Inc. in Seattle. My colleagues, Tessie Tzavaras Catsambas from EnCompass LLC and Rakesh Mohan who works with the Idaho Legislature’s Office of Performance Evaluations, and I are presenting at AEA’s upcoming conference on managing the sometimes conflicting agendas of evaluation stakeholders. (They both will be posting on aea365 soon.) We each have somewhat different clientele, but all come up against the struggle to balance divergent priorities and ensure we are responsive to our clients’ needs.
Most important in addressing the plurality of agendas is getting them out on the table. Without this clarity, evaluation work easily bogs down. People involved can go around and around about any number of issues without understanding why they can’t move ahead. How many times have we sat through meetings going over the same territory for the umpteenth time? What may be behind this is disagreement or lack of understanding about the purpose and goals of the evaluation, most important questions to answer, and use of the data. A few steps taken at the frontend can be a worthwhile investment.
Hot Tip: Include a goal clarification exercise in your project initiation process. It doesn’t have to be overly structured, perhaps just asking people around the table to identify the evaluation’s purpose from their perspective and the values they hold regarding the program — quality service, financial stability, cultural appropriateness, client satisfaction, community support, social justice. The goal is not necessarily to gain agreement, but to inform discussion about evaluation questions, selection of methods, and issues needing consideration. This leads to clearer understanding of the definitions of quality, value and effectiveness that become criteria for determining program performance.
Hot Tip: Use a theory of change visual to show links between program activities and assumptions and its outcomes, then involve key players in identifying priorities for measurement. This will make the work ahead both focused and visible to everyone.
Rad Resources: Jane Davidson’s presentation at the Aotearoa New Zealand Evaluation Association event in 2010, Visible Values: Striving for truth, beauty and justice in evaluation (http://realevaluation.com/pres/Visible-values-anzea-Dec10.pdf), provides a great conversation on making agendas and values transparent.
Hot Tip: Plan your data collection strategies to do double duty … look for opportunities to piggyback on each method. Add a question or two to surveys or interviews that will satisfy the needs of more than one stakeholder’s agenda. Use record or document review for multiple purposes. Build in multiple elements to any observation conducted.
Above all, be open and honest — act as a role model for others and build your credibility and trust among the group members.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Want to learn more from Dawn and colleagues? Attend their session “Whom Does an Evaluation Serve? Aligning Divergent Evaluation Needs and Values” at Evaluation 2011. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.