AEA365 | A Tip-a-Day by and for Evaluators

TAG | purpose

My name is Alberta Mirambeau and I am an ORISE fellow on the Evaluation and Program Effectiveness Team in the Division for Heart Disease and Stroke Prevention at the Centers for Disease Control and Prevention. I provide evaluation technical assistance to state-funded programs that implement heart disease and stroke prevention activities.

Our team uses the CDC Framework for Program Evaluation as our primary approach to evaluation. The Framework is organized into six steps, in which the first step is to “Engage Stakeholders.” During this step, typical questions asked are: How do you decide what stakeholders to include in an evaluation and to what extent do you involve them? For many programs, the list of program stakeholders — people or organizations that have an interest in a program — may be quite long. For example, in a service-delivery program, its funders, administrators, implementers, and participants may all have a perspective to add to the design and delivery of the program. The many viewpoints may lead to differences or conflict about what an evaluation should accomplish and how it should be conducted.

As an evaluator, I find a helpful way to address the issue of a large set of diverse program stakeholders is to make a distinction between program stakeholders and evaluation stakeholders. Stakeholders for an evaluation typically emerge as a core group from the program stakeholders. Evaluation stakeholders are the primary users of the evaluation results as well as those who will be involved in designing or implementing the evaluation. Evaluation stakeholders are called on to advise about program processes, the evaluation design, and the implementation of the evaluation. Although you may keep program stakeholders informed about the evaluation process and its progress, the evaluation stakeholders serve as key advisors. By identifying key advisors to guide the evaluation process, you’re building support for the evaluation and helping to ensure the utility of the evaluation.

Hot Tip: Before engaging any stakeholders in an evaluation, identify the purpose of the evaluation as well as the intended use and users of evaluation results. Allow the answers to these to guide your selection of evaluation stakeholders for a particular evaluation.

Hot Tip: Evaluation stakeholders may be different for each evaluation conducted. For example, an evaluation that focuses on program improvement may include program participants who may not need to be included in an evaluation that focuses on program administrative processes.

Rad Resource: The CDC Framework for Program Evaluation is available at http://www.cdc.gov/eval/ framework.htm.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

My name is Diane Dunet and I am a senior evaluator on the Evaluation and Program Effectiveness Team at the Centers for Disease Control and Prevention, Division for Heart Disease and Stroke Prevention. Our team members use a written purpose statement for our program evaluations.

In strategic planning, a mission statement serves as a touchstone that guides the choice of activities undertaken to achieve the goals of an organization. In evaluation, a purpose statement can serve as a similar touchstone to guide evaluation planning, design, implementation, and reporting.

Early in the evaluation process, evaluators on our team at CDC work with our evaluation sponsors (those requesting that an evaluation be conducted, for example a program manager) in order to understand and clarify the evaluation’s purpose. In many cases, the purpose of an evaluation is to improve a program. Other types of evaluation purposes include accountability, measuring effectiveness, assessing replicability of a program to other sites, determining what program components are essential, or making decisions about a program’s fate. We develop a written evaluation purpose statement and then refer to it during the entire evaluation process. An example purpose statement is:

The purpose of this evaluation is to provide an accountability report to the funder about the budgetary expenditures for client services delivered at 22 program sites. (Accountability.)

In the initial stages of evaluation, we are guided by the evaluation purpose when determining which program stakeholders should be involved in the evaluation in order to accomplish its purpose. We refer to the purpose statement to guide our evaluation design, seeking to match data collection methods and instruments appropriate to the evaluation purpose. We also use the evaluation purpose statement to guide us in tailoring our reports of evaluation results to align with the sponsor’s needs and the evaluation’s purpose.

Of course, evaluation findings can sometimes also be “re-purposed” to provide information in a way not originally intended, for example when program managers find ways to improve a program based on results of an evaluation for accountability.

Resource:  The CDC Framework for Program Evaluation in Public Health provides a six-step approach to conducting program evaluation and is available at http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm

Resource:  The CDC Division for Heart Disease and Stroke Prevention sponsors a public health version of “Evaluation Coffee Breaks” modeled after the AEA Coffee Breaks. Information and archived sessions are available at http://www.cdc.gov/dhdsp/programs/nhdsp_program/evaluation_guides/index.htm

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Archives

To top