Memorial Day Week: Defining Evaluation in the Military by Stephen Axelrad

Hello, my name is Stephen Axelrad – the founder and chair of the Military and Veteran Evaluation Topical Interest Group. 

One of the reasons why I wanted to start this TIG was to broaden the military’s understanding of what program evaluation is and broaden program evaluator’s understandings of what the military community is.  While there are pockets of the military that conduct evaluations from the health, behavioral, and social science perspective, the majority of program evaluation in the military is conducted from a resource management perspective.  In the Department of Defense and the military service branches, program evaluation is embedded in what is considered the program planning, budgeting, and evaluation (PPBE) process.  Although different terms are used, the goals of the PPBE process are very similar to what we would recognize in most evaluations.

1. Describe and understand the key assumptions, inputs, activities, outputs, and outcomes associated with a program, strategy, or initiative.
2. Identify areas of effectiveness and ineffectiveness, gaps, and innovations in current programs, strategies, initiatives, and systems.
3. Provide rigorous evidence to support quality decisions from leaders.
4. Use data to improve program planning and execution.

Hot Tip: Many program evaluators, especially those who exist outside of the military healthcare and community support agencies have financial management, operations research, and systems engineering backgrounds.  These fields understand and use some of the same methodological (e.g., surveys, interviews) and analytic techniques (e.g., regression, structural modeling) that evaluators within AEA frequently use.  Finding common ground on methodology can help avoid pedagogically based misunderstandings. 

Rad Resources: Before engaging with a project or initiative that involves DoD or military service branch officials, review the following web sites to become more familiar with how they may approach evaluation.

• Office of the Secretary of Defense’s, Cost Assessment and Program Evaluation

• Office of the US Army Deputy Chief of Staff, G8, Program Analysis and Evaluation Directorate

• Cornell University’s School of Operations Research and Information Engineering’s What is Operations Research?

Lesson Learned: Evaluators do not have to be in the military in order to understand how to design and execute evaluations affecting the military community.  Evaluators who demonstrate an understanding of the military decision making process (MDMP) and can structure their evaluations as closely as possible to MDMP can maximize buy-in from uniformed and civilian leaders and the use of their evaluation results.  The accompanying graphic visualizes the underlying logic behind MDMP.

The American Evaluation Association is celebrating Memorial Week in Evaluation. The contributions this week are from members of the Military and Veteran Evaluation TIG featuring contributions to evaluation with military origins but relevant to all we do. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.