Michael Duttweiler on Focusing an Evaluation

Hello, My name is Michael Duttweiler and I am assistant director for program development and accountability for Cornell University Cooperative Extension.  One of my roles is to provide evaluation planning assistance to local extension educators, many of whom do not have formal backgrounds in program development or evaluation.

Hot Tip: Focusing an Evaluation Using a Succinct Purpose Statement There are many resources that guide one through focusing an evaluation typically ranging through a series of considerations such as program life cycle, stakeholder interests, key questions to be answered, resource considerations, respondent access, evaluability, and the like.  Many people not steeped in evaluation practice find starting the planning process with such a broad array of considerations both complex and intimidating.  A number of years ago, I landed on the approach of starting first by asking people to draft a brief evaluation purpose statement. This is a one-paragraph description of their evaluation effort that describes what is and is not being evaluated and the purposes of the evaluation.  It sets boundaries by including a description of the program elements and time frame being considered, which audiences are being addressed, and which goals or objectives are of most interest.

Example: The purpose of this evaluation is to assess the effectiveness of the 2010 Master Forest Owners Workshop in supporting and prompting volunteers to extend their knowledge to other forest owners in their local communities.  A secondary purpose is to provide documentation and assessment information for use by persons considering replicating the model with other forest owner groups.  Considerations include program structure and processes, curricular choices, and short-term impact assessment.  Other means of supporting forest management volunteers such as our newsletter and quarterly conference calls will not be assessed.

The purpose statement becomes an anchor to which we return at each phase of the planning process modifying the statement as necessary and appropriate.  With a draft purpose statement in hand, I then move to identification of the one or two priority evaluation questions to be answered based on the purpose statement.  The usual timeframe, respondent access, and resource limitation questions can then be raised in context and with a clear sense of intent as expressed in the purpose statement and key questions.

This week’s posts are sponsored by AEA’s Extension Education Evaluation Topical Interest Group (http://comm.eval.org/EVAL/Extension_Education_Evaluation/Home/Default.aspx) as part of the EEE TIG Focus Week. Check out AEA’s Headlines and Resources entries (http://eval.org/aeaweb.asp) this week for other highlights from and for those conducting evaluations in an Extension Education context.