I’m Jim Burdine and a few weeks ago I posted a “tip-a-day” (see post here) on presenting community health assessment data back to the community. This is part two of that discussion. Last time I focused on “what to present” this posting focuses on “how to present . . . with results.”

There are two goals for any assessment presentation: (1) reporting the data, and (2) identifying next steps for improving health status. I’ve used a format that is effective in a variety of settings. This assumes, as discussed in my last posting, that the assessment has been a “participatory process.”

Hot Tip: Hold a “regional health summit.” Folks invited include those who have participated in various aspects of the assessment as well as others representing a broad cross section of community sectors. My experience has largely been in medium to smaller communities (500K to 10K population) but participation is roughly the same – about 150-250 folks is a good turnout. The event is split into two parts and can be accomplished in one day or two. The two parts are a presentation of the data, followed by a workshop. Ideally the data presentation is limited to an hour and the actual presentation, or much of it, is by a community member. Avoid the temptation to have more than a couple of speakers – “inclusiveness” is a nice idea but counterproductive in this context. You can invite community members to do “welcomes” and other remarks.

Following the data presentation is another opportunity to develop for community buy-in. In the workshop, we ask folks to select among the “findings” of the assessment (e.g., “access to mental health,” “childhood obesity,” “rural transportation”) and sign up for those topics. Have rooms/tables setup for each group. A convener appointed for each group opens the discussion and is responsible for encouraging folks to commit to participating in an ad hoc task group on this topic that that would agree to meet 2-3 times over the next 2-3 months. At the end of that time (3 months) the larger group is reconvened to hear reports from the various ad hoc task groups and then to endorse a regional health improvement strategy developed at that session (based on the group reports). During the 2-3 month work period, each team reviews pertinent findings and identifies “best practices” of interventions related to that issue. The group then identifies one or two priority activities that incorporate those best practices as adapted to your community.

By following this model, what might have been just a data presentation yields a comprehensive regional strategy with multiple sub-parts that is endorsed by the entire community (at least as represented by the individuals and agencies participating in the overall process).

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

communication, participatory

Hi, my name is Susan Sloan. I’m a program evaluator with about 20 years of experience—first as an evaluation team leader for Duerr Evaluation Resources in California and now as in internal program evaluator for Whatcom County Health Department (WCHD) in the Pacific Northwest.

As a small local health department, we are always looking for ways to increase internal evaluation capacity without adding additional staff. If you’ve been reading the news lately, you’ll know that local public health resources are decreasing at an alarming rate. This makes it even more important that the programs we run are effective and that our remaining staff is trained to understand evaluation and to participate as part of an evaluation team when needed.

In order to improve organizational evaluation capacity, I’ve used the Centers for Disease Control and Prevention’s Framework for Program Evaluation in Public Health as a teaching tool (MMWR 1999; 48 (No. RR-11): http://www.cdc.gov/eval/framework.htm

When I first discovered CDC’s Framework, I was amazed at how well it mimicked the evaluation process I had used for years to evaluate school intervention programs. The best feature of the framework is that it is an easy-to-understand, easy-to-teach six-step process for evaluation. Here at WCHD, we used our Community Health staff meetings to teach the framework over a six-month period. In order to make the training come alive, we used examples from an in process evaluation of our Children with Special Health Care Needs (CSHCN) program along with a staff-created evaluation of a mythical public health trails infrastructure campaign. The culminating activity resulted in a short report that was written by staff.

Hot Tip: The first AEA Coffee Break focused on DoView®, a modeling software developed by Dr. Paul Duignan: http://www.doview.com/ We have purchased several copies of this software as a wonderful augment to our use of the CDC Six-Step Framework. We are now able to create evaluation models that work us through the framework. Our DoView® models include: (1) a program overview, including  overall goals and major program components, (2) a comprehensive listing of all internal and external stakeholders, (3) a flow chart of each major program component, (4) a logic model, (5) an evaluation design (including the evaluation mission, major questions, methods, assignments, and timelines), and (6) reporting of evaluation findings. All of this can be easily shared with team members or partners either through DoView files, pdf’s, or HTML documents.

The CDC Framework combined with the DoView® software has allowed us to create an evaluation toolkit that meets the ever-challenging needs for local health department evaluation capacity building.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

CDC, evaluation capacity, logic models, Public Health

I’m Jim Burdine, Assistant Dean for Public Health Practice and Co-PI/Director of a Prevention Research Center at the School of Rural Public Health, Texas A&M. Over the last 30 years I’ve used community health status assessment as both a community organizing tool and intervention planning tool. In more than 200 different communities (including multiple iterations in the same community) I’ve presented/seen data presented from community assessments in a number of different formats with varying degrees of success and failure.

Lessons learned: What I’ve observed as most important, is presenting the data to them in a manner that matches the expectations of the audience – audiences, really. In other words, the groundwork you’ve already done in your assessment process (hopefully incorporating community-based participatory research principles) should dictate the format. If community members have been involved in planning, conducting and analyzing the data, they should play the major role in presenting the results. If they have been more passively involved, they may expect a “report” FROM you. Obviously the degree of “buy in” to the findings varies dramatically as a function of the degree of participation.

Given that starting place, the next challenges you face are: (1) the sheer volume of information you have to present, and (2) the variation in sophistication around understanding data within a community audience. So first, you have to accept that you can’t present EVERYTHING. You need to decide what are the key points you want to make and focus on those. We’ve all sat through a presentation where somebody reads us the demographics of a community or lists every chronic disease ever found in that population and an hour later you’ve learned nothing new. As a general rule I don’t both to report anything unless it is (1) statistically significantly different from some external reference point (e.g., Healthy People 2010, a state or national rate) and (2) unless there is something that could likely be done locally to impact that problem (it’s actionable).

If well-planned, you will have representatives from all community sectors in your audience (e.g., health care, business, elected officials, religion, education, the media, consumers and representatives of special interests/special needs groups). So you have to decide on what common denominators (e.g., educational attainment, exposure to health statistics) you are going to assume for your audience. You need to be comfortable with knowing that some aren’t going to understand everything you say and other are going to be bored with your “simplistic” presentation. Don’t make the mistake of trying to explain every point to each group n your audience. It just frustrates them and makes for a very ineffective presentation. Plan to do multiple presentations for different audiences rather than a “one size fits all” presentation.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

community, presentations, Public Health

My name is Susan Kistler. I am AEA’s Executive Director and I contribute each Saturday’s aea365 post. If you didn’t have the opportunity to join your colleagues at the AEA/CDC Summer Evaluation Institute this past month, you missed some great sessions  - the agenda may be found online.

Hot Tip: Over 80% of the presenters have shared the handouts in the AEA Public eLibrary! These materials, ranging from slides to bibliographies, to resource lists, are available to anyone to kickstart your own learning (and perhaps whet your appetite to join us next year in Atlanta – I’ll keep you informed about dates as they are finalized via aea365).

• Bringing Funders, Researchers/Evaluators, and Practitioners Together to Achieve Outcomes: The Interactive Systems Framework (ISF) for Dissemination and Implementation (Wandersman & House)
• Conducting and Using Success Stories for Capacity Building (Lavinghouze & Price)
• The Democratization of Data Insight: 20 Free Tools for Data Access, Analysis, and Reporting and Considerations for Their Use (Kistler)
• Dissemination and Implementation Research (Emshoff & Noonan)
• Enhanced Group Facilitation: Techniques and Process (Dewey)
• Evaluating Environmental Change Strategies… Beyond Counting the Number of Laws Passed (Pettibone)
• Every Picture Tells a Story: Flow Charts, Logic Models, LogFrames, Etc. What They Are and When to Use Them (Chapel)
• Handling Data: From Logic Model to Final Report (Barrington)
• How Policy Is Made and How Evaluators Can Affect It (Grob)
• Improving Survey Quality: Assessing and Increasing Survey Reliability and Validity (Germuth)
• An Introduction to Economic Evaluation (Corso)
• Introduction to Evaluation (Apt)
• Leading Concepts in Community Health Evaluation (Goodman)
• Lessons From the Field: Ways to Improve Your Evaluation Practice (Barrington)
• Logic Models as a Platform for Program Evaluation Planning, Implementation, and Use of Findings (Honeycutt)
• Methods for Analyzing Change Over Time (McKnight)
• Minding Your Mind (Grob)
• Performance Management in the Obama Administration (Newcomer)
• Program Evaluation and Program Improvement: Challenges, Opportunities, and New Directions at the Centers for Disease Control and Prevention (Collins)
• RealWorld Evaluation (Rugh)
• Qualitative Evaluation Approaches (Goodman)
• Quantitative Methods for Evaluation (McKnight)
• Rethinking Social Change Using Social Media (Fine)
• Strengthening Evaluation Through Cultural Relevance and Cultural Competence (Kirkhart & Hopson) also available here
• Systems Level Evaluation of Communities of Practice (Hegedus & Jernigan)
• Transformative Mixed Methods Evaluation (Mertens & Bledsoe) Plus list of references
• Translation of Evaluation Capacity Building Strategies to Community-Based Organizations Conducting Health Promotion Activities: Tools, Tips and Lessons Learned (Akintobi)
• Using GIS in Evaluation (Meyers)
• Using Program Evaluation to Improve Nonprofit Outcomes (Newcomer)
• Using the Guiding Principles to Improve Your Evaluation Practice (Goodyear)
• Writing Questions that Elicit What You’re Looking For (Pohl)

No tags

Hi!  My name is Hazel Atuel and I am the Research Program Director at the Comprehensive San Diego State University/UC San Diego Cancer Center Partnership. Today I will be sharing some hot tips and rad resources on community competency.

I am grateful to Dr. Bob Robinson, Associate Director for Health Equity of the Office on Smoking and Health, National Center for Chronic Disease Prevention and Health Promotion (CDC, Ret), for sharing his Community Model to Eliminating Health Disparities. Dr. Robinson’s theoretical framework is quite comprehensive and I would like to highlight the various dimensions of community competency, one of the major components of the model, as they will serve us well in the work we do.

To illustrate the need for community competency, let me share one of Dr. Robinson’s stories. When a CDC-funded REACH program in a northeastern state conducted a study almost a decade ago, data collection involved recruiting Cambodian refugees to participate in the project. As the study progressed, reports from the field reflected almost nil success in engaging participants from this target population. Only after someone provided a cursory review of Cambodian history did the program leaders understood where the resistance stemmed from: Asking people to “participate in a study” was one of the strategies the Khmer Rouge used to entice people from the city to go to the rural areas that led to the killing fields.

Had the researchers done their historical homework prior to collecting the data, recruitment strategies would have been very different. How then do we move forward as community competent evaluators?

Hot Tip: A first step is to differentiate clearly between community competency and cultural competency as the two constructs are not synonymous to each other. While cultural competency focuses on the individual, the unit of analysis in community competency is the community or group.  Second, the primary constructs of community competency are history, culture, context, and geography, and the secondary constructs are language, literacy, positive and salient imagery, multigenerational appeal, and diversity (Robinson, 2005, pp. 339-340). I refer the reader to the Rad Resource for in-depth reading even though some of these constructs are self-explanatory. I think it wise to generate a community competence checklist so we can be better equipped as evaluators for programs that serve diverse communities.

RAD RESOURCE: Robinson, R.G. (2005).  Community Development Model for Public Health Applications: Overview of a Model to Eliminate Population Disparities. Health Promotion Practice, 6, 338-346.

Robinson, R., and Holliday, R. (2009). Tobacco-use and the Black Community: A Community-focused Public Health Model for Eliminating Population Disparities. In R. Braithwaite, S. Taylor, and H. Treadwell, H. (Eds.), Health Issues in the Black Community, Jossey Bass: CA (pp. 379 – 416).

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

community, competency, Public Health

I’m Tom Chapel, the Chief Evaluation Officer (acting) at the Centers for Disease Control and Prevention. I’ve also served as co-chair of the AEA/CDC Summer Evaluation Institute since its inception.  The Institute turns 10 years old this June and, with that in mind, I wanted to share a lesson learned and a couple of great resources for evaluators.

Lesson learned: Understand the difference between “primary” and “secondary” demand for a product or service.

• Primary demand – milk is good for you… Got milk?
• Secondary demand –buy [brand] milk because it is locally farmed/cheap/vitamin-reinforced etc.

This simple, fundamental marketing principle has import for evaluators.  The primary demand we draw from is the desire of programs to make an impact, understand their program, report out success.  Sometimes that secondary demand plays out as what we recognize as “program evaluation” but just as often, the relevant product/approach is performance measurement, quality assurance, or strategic planning.  Sometimes, I’ll be leading a leadership meeting for one of our programs.  I will be creating some simple logic models with them, but deploying those to affirm mission/vision and make some strategic decisions.  The word evaluation may not even come up.  But being present and involved in that conversation, and using key tools in my evaluator arsenal, I know I’m setting them up for strong evaluation later.  By reframing our thinking as evaluators so that we talk about organized reflection on a program—whether processes or outcomes—we reinforce the idea of continuous program improvement and the integration of planning, performance measurement , and evaluation.

Two useful resources from CDC:

Resource: CDC’s Framework for Program Evaluation, while originating within public health, is broadly applicable in many contexts and reinforces the idea of use of findings for program improvement. The Framework’s website provides a detailed explanation of the framework as well as multiple resources that support its implementation.

Resource: CDC’s National Center for Injury Prevention and Control just released Evaluation for Improvement: A Seven-Step Empowerment Evaluation Approach. This manual is designed to help violence prevention organizations hire an empowerment evaluator who will assist them in building their evaluation capacity through a learn-by-doing process of evaluating their own strategies. But any organization considering empowerment evaluation may also find it valuable.

Hot Tip: If you want to enhance your evaluation-related knowledge and skills, join over 500 of your colleagues at the AEA/CDC Summer Evaluation Institute. This is our 10^th year; we welcome attendees from any discipline to Atlanta from June 13-16 for professional learning and networking.

CDC, CDC Framework, Demand, institute, Public Health