AEA365 | A Tip-a-Day by and for Evaluators

TAG | SWOT Analysis

Hi—we are Garry Lowry from the Office for State, Tribal, Local and Territorial Support at the Centers for Disease Control and Prevention, Lynn Chaiken from the Association of State and Territorial Health Officials, and Tamara Lamia from ICF International. We are part of a team that conducted a rapid eight-month evaluability assessment of the Preventive Health and Health Services Block Grant. The Block Grant provides all 50 states, the District of Columbia, two tribes, and eight US territories with flexible funding to address their unique and emerging public health needs.

Because block grants and flexible funding mechanisms are, by definition, broad, you might find that you improve a program’s evaluability by doing the assessment itself, especially when attempting to describe the program. We present some hot tips and cool tricks for using evaluability assessment tasks to clarify the program definition before doing more formal evaluation.

Hot Tips: Draft or Refine Program Goals, Objectives, and Strategies. After reviewing Block Grant resource documents and engaging diverse stakeholders, we created an overarching goals, objectives, and strategies document that facilitated a shared understanding of the program’s intended outcomes and how it planned to achieve them. When a basic program definition is lacking or not explicitly articulated, creating one can help make post-assessment evaluation planning easier.

Draft or Refine a Program Logic Model. Once we had broad stakeholder acceptance of the goals, objectives, and strategies, we drafted a logic model with stakeholder input. The logic model demonstrated how the identified strategies led to the program’s objectives and goals. In our case, creating a logic model after drafting and gaining stakeholder buy-in on explicit program goals, objectives, and strategies minimized the necessary back and forth with our large stakeholder group. Adding a logic model provides a graphic description of your program and forms a foundation for evaluation planning.

Cool Tricks: Leveraging a SWOT Analysis. A SWOT analysis helps identify the strengths, weaknesses, opportunities, and threats of a program or course of action. In the evaluability assessment context, the SWOT analysis gave us a framework for thinking about program aspects that might help or hinder future evaluation efforts.

Using a Visual Dot Prioritization Exercise. We used a visual dot prioritization exercise to prioritize numerous outcomes for short- and long-term evaluation. With the outcomes posted on flipcharts around the room, each stakeholder placed one colored dot on the most important outcome to evaluate and a different colored dot on the most feasible outcome to evaluate in the near term. The result was a visual of colored dots, which we sorted in a 2×2 matrix to analyze which outcomes were deemed the most important and most feasible for immediate evaluation.

We’re celebrating 2-for-1 Week here at aea365. With tremendous interest in the blog lately, we’ve had many authors eager to share their evaluation wisdom, so for one special week, readers will be treated to two blog posts per day! Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

I am Jackson Hille, the Content Associate for FormSwift, a SF-based startup that helps organizations, entrepreneurs, and businesses go paperless. Recently, for a work project, I had to evaluate the current strengths and weaknesses of our content campaigns, and also decide what content avenues present our greatest opportunities and our greatest threats. As a professional evaluator, you might be familiar with the aforementioned planning concept, commonly referred to as a SWOT Analysis. I am new to the business world, as I just graduated from UC Berkeley in May 2014, so as any decent millennial would do, I Googled and searched guides to SWOT Analysis. While the majority of what I found was disappointing, there was one existing rad resource out there from the University of Kansas; nonetheless, there was no SWOT Guide that combined honest evaluations about the modern economy, usable templates and a user friendly design. So, I called up my college mentor and we created our own, “Essential Guide to SWOT Analysis.”

Rad Resource: The Community Tool Box’s SWOT Chapter:

The Community Tool Box is a service of the Work Group for Community Health and Development at the University of Kansas. Their SWOT Guide is comprehensive, user friendly and a great resource for any evaluator within the public health or community development field. The SWOT Guide not only provides a thorough explanation of the appropriate evaluation process when conducting a SWOT Analysis centered around health and community development related issues, but it also provides readers with multiple forms of content. For instance, it contains relevant examples for health and community development professionals, a ready-made PowerPoint version of the guide’s highlights, and a handy checklist to ensure you are taking the correct steps when conducting your SWOT Analysis.

Rad Resource: The Essential Guide to SWOT Analysis:

The Essential Guide to SWOT Analysis is the end product of a unique collaboration. As previously noted, through my work at a startup, I realized that there were no great, comprehensive SWOT guides out there, especially for people in the business world. So, I enlisted my college mentor, Justin Gomer, a Lecturer at UC Berkeley, and we decided to make a comprehensive guide ourselves. The finished product is a guide to SWOT Analysis that is readily usable for either a professional evaluator hired as a consultant to help out with a company’s investment strategy, or for a volunteer at a non-profit, who needs help evaluating an organization’s goals for the upcoming summer.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

I’m Bethany Laursen, Evaluation Outreach Specialist with the Solid & Hazardous Waste Education Center (SHWEC) at the University of Wisconsin. I’m also principal consultant at Laursen Evaluation and Design, LLC. At SHWEC, I help staff design programs that engage opportunities to achieve our mission. Opportunity hunting requires a form of situation assessment, which has not been widely or deeply discussed in evaluation—especially when it comes to evaluating opportunities in complex, dynamical situations.

Rad Resource: AEA’s EvalTalk and TIG group listservs as peer learning communities.

Through EvalTalk, several colleagues helped me distinguish among three approaches/tools that all claim to be useful in developing programs in complex situations: needs assessment (NA), developmental evaluation (DE), and strengths, weaknesses, opportunities and threats (SWOT) analysis.

Lesson Learned: NA, DE and SWOT are all necessary parts of evaluating complex situations and program responses.

To summarize this discussion so far, we have the following options, where () means “as a part of” e.g. NA is a part of SWOT:

  1. NA–>SWOT–>DE
  2. SWOT(NA)–>DE
  3. NA–>DE(SWOT)
  4. DE(NA, SWOT)

Any of these combinations is logical, although #4 might be difficult without one of the others occurring first. What is not logical is leaving one of the triumvirate out (NA, DE, and SWOT). Here’s why:

SWOT is inherently evaluative: it assigns data a certain value label (S, W, O, or T), based on the criteria “effect on our organization’s goals.” Clearly, we need data to do a reality-based SWOT, and this is why we must include a needs assessment. But a NA per se is not going to be enough data, because many clients think a NA is just about external stakeholders’ needs (Os), not internal capacity (Ss and Ws) or larger system realities (often Ts). (If preferred, one could also frame a NA as an ‘asset assessment.’) These external and internal ‘lessons learned’ from our situation should inform developmental program evaluation.

In complex situations, needs assessment is more usefully framed as ongoing situation assessment. This is what I see as the main evaluation task in the Creative Destruction phase of the adaptive cycle. Once we have a lay of the land (situation assessment) and we’ve evaluated the best path to start down (SWOT analysis), then we can jump into developmental evaluation of that path. Of course, what we find along the way might cause us to re-assess our situation and strategy, which is why #4 above is a logical choice.

Lesson Learned: Listen to the language your clients are using to identify relevant evaluation approaches and tools. In SHWEC’s case, our connection to the business sector led me to SWOT analysis, strategic planning, and Lean Six Sigma, all of which are evaluative without necessarily marketing themselves as evaluation approaches.

Figure 1: Augmenting a traditional logic model, this is a metaphorical picture of how SHWEC understands our complex, dynamical situation and our potential evaluation questions. (Each sailboat is a staff member.) Next, I had to find evaluation approaches that would fit.

Figure 1: Augmenting a traditional logic model, this is a metaphorical picture of how SHWEC understands our complex, dynamical situation and our potential evaluation questions. (Each sailboat is a staff member.) Next, I had to find evaluation approaches that would fit.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hello! I’m Michelle Baron, an Independent Evaluation Strategist. In my work in higher education, I’ve encountered a mixture of evaluation champions and critics. Today I’d like to address the importance of strategic planning in building a culture of evaluation.

Strategic planning is considered by many to be an organizational road map by outlining the organizational vision and mission, establishing clear and attainable objectives and goals, and then developing processes for how to achieve them.    Strategic planning and evaluation go hand in hand in moving the organization and its programs forward to benefit its stakeholders. Strategic planning is simply crucial to the evaluation process: without a road map of criteria, standards, and goals, it’s almost impossible to achieve desired success.

Evaluators have a unique role in helping organizations with both ends of the spectrum: creating a foundation through strategic planning, and then conducting evaluations to examine and monitor progress.

Hot Tip #1: Start at the top. Buy-in from top management for strategic planning is of the utmost importance for its success.

Hot Tip #2: Conduct a SWOT analysis (Strengths, Weaknesses, Opportunities, and Threats) of the entity or its programs/services. Doing so not only enlightens people to a variety of ideas and questions to consider, but can also indicate the level of support for those topics.

Cool Trick: Brainstorming sessions are often an excellent starting point for the organization itself or smaller group within that organization. The evaluator or designated member of the organization can facilitate the discussion by developing questions beforehand that may serve as prompts for the discussion, such as those dealing with objectives, goals, and resources.

Rad Resource #1: Strategic Planning for Public & Nonprofit Organizations by John Bryson, and related books by the same author, provide the groundwork and tools necessary for organizations to develop and sustain their strategic planning process.

Rad Resource #2: The Fifth Discipline: The Art and Practice of the Learning Organization by Peter Senge helps leaders establish the foundation and philosophy behind strategic planning, and helps them develop their long-term thinking for organizational growth.

With these tools and resources, evaluators may be more prepared to assist organizations in strategic planning, and have more support for and effectiveness of the evaluations for the organizations.

The American Evaluation Association is celebrating Business, Leadership and Performance (BLP) TIG Week with our colleagues in the BLP AEA Topical Interest Group. The contributions all this week to aea365 come from our BLP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

· · · ·

Archives

To top