AEA365 | A Tip-a-Day by and for Evaluators

CAT | Community Psychology

Hi! I am Cathy Lesesne and I work at ICF International doing public health related evaluation and research. My passion is doing work that affects the lives of adolescents, particularly those with the most need and the least voice in how to meet those needs. I do a lot of work in and with schools and school districts focused on optimal sexual health for teens and how to ensure youth have skills and ability to make healthy choices no matter when they decide to engage in sexual activity.

I often see well-intentioned school or school district staff creating solutions for youth and testing them rather than involving youth in solution identification and evaluation of the success. It is clearly easier to retain the power to determine the solutions and to see if they work in the end through evaluation. However, in my own work I have seen the power of youth engagement and involvement in both developing programs and services as well as in helping to evaluate and improve those resources.

Rad Resources: As evaluators, we often have the ability to make recommendations to our clients and partners working with youth AND we have the power to approach our evaluation work with youth in empowering and engaging ways. But we don’t always know how. I highly recommend that you dig into the Youth-Adult Partnerships in Evaluation (Y-AP/E): A Resource Guide for Translating Research into Practice and find your own ways to apply the wide range of ideas, tip sheets, and examples for engaging youth as partners in evaluation. Many of these examples may also help your clients or partners think of ways to better engage youth in the development of programs and services that reflect them and their real interests and needs. If youth are empowered to be partners in developing and testing solutions, they become allies instead of subjects; sources of solutions instead of sources of data.

The American Evaluation Association is celebrating Community Psychology (CP) TIG Week with our colleagues in the CP AEA Topical Interest Group. The contributions all this week to aea365 come from our CPTIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

I am Carrie Lippy, an independent evaluator working with community-based and culturally specific domestic violence agencies. For the last two years, I have worked closely with the NW Network, an agency providing intervention and prevention services to lesbian, gay, bisexual, transgender, and queer (LGBTQ) survivors of abuse.

Thinking about community-level impacts often brings to mind regional or place-based notions of community. For example, interventions targeting neighborhoods, cities, or counties. However, many culturally specific programs aim to impact identity-based, rather than place-based communities. Identity-based communities are those developed among people with shared identities, such as sexual orientation or gender identities.

Below are some lessons learned & rad resources for evaluating community-level impacts for identity-based communities.

Lessons Learned:

  • Be clear on who the community is. Defining identity-based communities can be tricky. For example, when looking at impacts on LGBTQ communities, evaluators need to be mindful of the impressive diversity of LGBTQ people, recognizing that even the terminology used to identify members of these communities may differ widely. Terminology can differ by factors such as age, race/ethnicity, and region. Some members of LGBTQ communities may even identify as heterosexual (e.g., some transgender people).
  • The importance of online community spaces for identity-based communities. Since identity-based communities typically have less connection to geographic areas, online spaces hold particular importance for connecting community members. In fact, even online spaces that are not culturally specific can still reach many identity-based communities. Recently, the National LGBTQ Domestic Violence Capacity Building Learning Center partnered with the National Domestic Violence Hotline to examine the experiences of LGBTQ survivors of domestic violence. Although the online survey was posted on the non-LGBTQ-specific Hotline website, nearly 600 LGBTQ survivors completed the survey, illustrating the reach of even non-culturally specific online spaces.
  • A need for alternative sampling strategies. Some identity-based communities can be especially challenging to reach, making measuring community-level effects quite difficult. In my work with the NW Network, we’ve found snowball sampling strategies particularly effective for reaching marginalized members of LGBTQ communities, including some transgender communities of color or LGBTQ immigrants. Snowball sampling techniques utilize existing connections in communities to recruit research participants.

Rad Resources:

For those interested in learning more about culturally-specific research and practice in LGBTQ communities, check out:

  • A free, online library with resources on domestic violence in LGBTQ communities. The library was created by the National LGBTQ DV Capacity Building Learning Center, a joint project of the NW Network and the National Coalition of Anti-Violence Projects.

The American Evaluation Association is celebrating Community Psychology (CP) TIG Week with our colleagues in the CP AEA Topical Interest Group. The contributions all this week to aea365 come from our CPTIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hi! My name is Amy Hilgendorf and I am the Associate Director for Engaged Research at the University of Wisconsin-Madison Center for Community and Nonprofit Studies (the CommNS). We specialize in community-based action research and evaluation partnerships with grassroots and nonprofit groups and offer support to others who do this work.

In recent years, we have partnered with county-based and statewide coalitions that are seeking to address childhood obesity by applying a model of collective impact. John Kania and Mark Kramer first characterized collective impact as consisting of five key conditions that can help unite multi-sector collaborative efforts towards greater community impact than what isolated efforts can achieve. Those five conditions are: a common agenda, mutually reinforcing activities, continuous communication, shared measurement systems, and backbone support. The coalitions we work with have found the collective impact model offers valuable guidance for the kinds of processes that will set them up for achieving impact, but questions remain about how to actually evaluate the impacts of collective impact.

Rad Resource:

The Collective Impact Forum is an online hub of information, resources, and peer networking related to collective impact. The searchable resources section includes a host of “Evaluation” resources. One tool is the Guide to Evaluating Collective Impact by Hallie Preskill, Marcie Parkhurst, and Jennifer Splansky Juster. While much of this guide focuses on evaluating the process of collective impact, the third part lists suggested behavior changes and systems changes that may result from collective impact initiatives and provides ideas of indicators and approaches for evaluating these changes.

Lessons Learned:

We have found it critical to remember that collective impact is not necessarily a new concept, but rather one that has emerged from a long tradition of collaborative and coalition practice and thinking. Literature on this topic stretch back more than 30 years, especially in the community psychology field, and includes theory and practical tools for assessing the process and impact of collaborative work.

In particular, the Community Coalition Action Theory developed by Fran Butterfoss and Michelle Kegler synthesizes much of this research to suggest how coalition practices can lead to different kinds of community impacts. These theorized impacts include community change outcomes, such as policy achievement and program expansions; community capacity outcomes, like new skill development and new partnerships; and, over time, the health and social outcomes that are the target of the coalition’s work. Additionally, we have found that Michelle Kegler and Deanne Swan’s efforts to empirically test the relationships in this theory offers especially useful guidance for “connecting the dots” between evaluation of coalition processes, including implementation of collective impact practices, and evaluation of community impacts.

The American Evaluation Association is celebrating Community Psychology (CP) TIG Week with our colleagues in the CP AEA Topical Interest Group. The contributions all this week to aea365 come from our CPTIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

My name is Courtney Barnard and I am a social worker, coalition coordinator, and program evaluator for a children’s health care system in Fort Worth, Texas.

Lessons Learned: I recently attended a workshop on local children’s health data and racial equity. I quickly realized that, though with good intentions, my colleagues and I often conflate race and ethnicity with other factors, such as poverty, education, and health status. In my community these factors strongly correlate with one another. Not only was I making assumptions about people and data based on these correlations, I was not intentionally analyzing outcomes by race. I was left wondering the impact of the consequences on my data analysis, recommended and implemented strategies, and ultimately the community affected by these decisions.

The Race Matters Institute of Just Partners, Inc. (RMI) seeks to ensure that ALL children, families, and communities thrive. When evaluating the effects of our efforts on the community, we must purposefully do so through a racial equity lens. Using this lens allows us to identify how various factors may affect racial groups with different resources and needs, thus shaping our analysis, recommendations, and strategies.

According to RMI, racial equity results when you cannot predict an outcome by race and when data reveals closing gaps in outcomes. It is quantifiable and measurable. When evaluating community impact through a racial equity lens, we must note:

  1. Attention to racial equity is key to advancing the mission for all community members, unless the stated mission explicitly targets a certain race or ethnicity.
  2. Applying a racial equity lens requires that data be broken apart by race. We must collect data by race and then systematically disaggregate the data in a consistent manner.
  3. Racial equity is about our shared fate as a community. It requires all voices and perspectives from the community be represented, valued, and included in policies and practices.

Rad Resource: The Annie E. Casey Foundation developed the Racial Equity Impact Analysis (REIA), a 5-question tool that can be used to assess any policy, program, or practice (PPP), either existing or proposed. This tool can help identify alterations in said PPP in order to achieve greater results for all community members.

  1. Are all racial/ethnic groups who are affected by the policy/practice/decision at the table?
  2. How will the proposed policy/practice/decision affect each group?
  3. How will the proposed policy/practice/decision be perceived by each group?
  4. Does the policy/practice/decision worsen or ignore existing disparities?
  5. Based on the above responses, what revisions are needed in the policy/practice/decision under discussion?

The American Evaluation Association is celebrating Community Psychology (CP) TIG Week with our colleagues in the CP AEA Topical Interest Group. The contributions all this week to aea365 come from our CPTIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

My name is Brian Hoessler, and I am the founder and principal of Strong Roots Consulting in Saskatoon, Canada. Actually, my business cards refer to me as Connector-in-Chief, because that’s how I see my evaluation and research work with non-profits: connecting questions with data, dreams with designs, and plans with reality. I also see my role as helping connect individuals, groups, organizations, and institutions towards a common goal, that of creating better communities for everyone. These two approaches are rooted in my community psychology background – one that emphasizes both an applied approach to social research and working with communities to help address local issues and promote social justice.

Evaluation and community psychology, as applied social research fields, share numerous commonalities: at the same time, they have a lot to offer each other. I hope this post will inspire evaluators to learn a bit more about community psychology, and community psychologists to engage more with the evaluation field!

Hot Tip:

What do collaborative approaches to change (e.g. Collective Impact), participatory and empowering methodologies, and systems thinking have in common? These concepts, growing in popularity in evaluation, were something that I first learned about through community psychology. Community psychology also brings an explicit values focus and a critical perspective, asking not just what’s happening but who’s benefiting and who’s marginalized: for example, Tom Wolff recently shared an engaging critique of the Collective Impact model.

Lesson Learned:

I found myself in the evaluation field initially for the professional development, but what’s kept me engaged are the people. I’ve found that evaluation brings together a diverse range of people from different backgrounds and experiences, including those who wouldn’t identify as an evaluator: instead, we come together out of common interests and purposes. With growing discussions around evaluation’s role in social change, sustainability, and global issues, it’s a field that community psychologists can and should be engaged in!

Rad Resource:

To learn more about the intersection of community psychology and evaluation, check out the Forum on Community Psychology in the American Journal of Evaluation’s March 2015 issue (Volume 36, Issue 1).

For more about community psychology, check out the Society for Community Research and Action (SCRA), the professional home for community psychology.

The American Evaluation Association is celebrating Community Psychology (CP) TIG Week with our colleagues in the CP AEA Topical Interest Group. The contributions all this week to aea365 come from our CPTIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Greetings. My name is Tiffeny Jimenez and I identify first as a Community Psychologist (CP). I am also an Assistant Professor of the Community Psychology Doctoral program at National Louis University in Chicago. As a CP, I inherently very quickly identify inequalities, injustices, and potential for collaboration where others may be more likely to see only conflict, and from this perspective, evaluation is a particularly salient and necessary skill set. How else might we judge whether social justice is achieved? It is towards this aim that I take on all inquiry and action. This year, I have had the privilege to be one of this year’s MSI Fellows where I have worked with colleagues towards gaining an in-depth interdisciplinary perspective on the state of our understanding Cultural Competence across Social Work, Health Psychology, Sociology, and Community Psychology. I will speak to the contribution of CP to this focus area.

Lesson Learned: The overall CP framework facilitates cultural competency and humility in all acts of professionalism with explicit emphasis on how we think and why we act in certain ways within a socio-cultural ecological context. CP views cultural competency as cross-cultural awareness assuming we all are interdependent and come to the table with diverse cultural lenses that influence action. Cultural competency is a critical consciousness beyond the acquisition of skills; it’s a way of being in every day interactions that allows for a clearer understanding of one’s own personal place in the world, personal biases, and an understanding that multiple perspectives are present at any one time.

The emphasis of CP is on promoting social justice and identifying the root causes of social problems by changing conditions so diverse populations can thrive individually within a shared geography. Much of the literature on cultural competency centers around: providing in-depth localized case examples of how CPs engage as equal peers with others to address individual and social problems from a culturally grounded perspective; describe adaptations of community programming to meet the needs of underserved populations; discuss the importance of using methods that capture historical context and the voices of less dominant perspectives; emphasize the promotion of dynamic processes within community-level systems rather than individual-level outcomes; and advocate understanding the cultural landscape that undergirds the various policies and practices that perpetuate inequalities and maintain the status quo. Main concepts: power, privilege, structural inequality, decolonizing methodologies, organizational culture, critical consciousness, liberation, indigenous psychologies, divergent cultural practices, and ecological sustainability.

Rad Resources: For more information on being culturally competent, see “Community Psychology: Foundations for Practice” (2015). Particularly Chapter 4 by Kien Lee titled “Effecting Social Change in Diverse Contexts”. The Global Journal of Community Psychology Practice also addresses cultural competency in CP practice from a global perspective: http://www.gjcpp.org/en/article.php?issue=16&article=77

The American Evaluation Association is AEA Minority Serving Institution (MSI) Fellowship Experience week. The contributions all this week to aea365 come from AEA’s MSI Fellows. For more information on the MSI fellowship, see this webpage: http://www.eval.org/p/cm/ld/fid=230 Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Greetings readers!  We are Norma Seledon and Susan Ryerson Espino and we are members of the Community Psychology (CP) TIG. Over the past two years, the CP TIG has organized a “Walk the Talk” session at the AEA annual meeting. This is a way to provide members with opportunities to learn more about the conference locale and practice the application of CP principles (such as community engagement and empowerment to evaluation work). In these sessions, we have visited neighborhoods and local community organizations, learned from community members and each other, and provided our host organizations with evaluation support.  

RAD RESOURCE:  Please join us November 12, 2015 for a toxic tour in La Villita.  We will better understand how local activists and community members have been engaged in community environmental justice action to raise awareness and reduce health disparities in Chicago.

La Villita is a historic port of entry for many Latino immigrants, particularly those from Mexico. Poor housing stock and transportation, toxic environmental elements including land, water, and air, dangerous work conditions, and dense neighborhoods with little green space all add to the health disparities impacting the everyday lives of La Villita’s working-class families. The rates of asthma, obesity, teen pregnancy, and poor mental health outcomes in La Villita are among the highest in the city of Chicago.

Staff from The Little Village Environmental Justice Organization (LVEJO) will guide us through their community.  They will teach us about their activism around environmental justice, criminalization, and gentrification. They will share how they have used culturally relevant popular education and other participatory tactics to engage community members.  We will hear about their powerful environmental justice struggles and witness firsthand how they have reclaimed toxic industrial sites to collaboratively design and enjoy healthy green space. A facilitated conversation will follow on how community psychology-informed evaluation can bolster grassroots action and program refinements.

Join us and other CP TIG members in this mutually-beneficial experience about how evaluation can be used to better serve diverse populations and help reduce health disparities in communities like La Villita. The walk starts at the Hyatt Regency front entrance at 2:00 pm on November 12th. Look for cars with posters stating “Community Psychology and Evaluation Community Tour.”  After the tour we will have a guided discussion, dinner, and the CP TIG business meeting.  We welcome all conference attendees to stay on for all activities.

The American Evaluation Association is celebrating CP TIG Week with our colleagues in the Community Psychology Topical Interest Group. The contributions all week come from CP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hello. My name is Chris Michael Kirk and I am the Director of Mission Development at Atlantic Health System in New Jersey where I lead our Center for Population Health Sciences. At the Center, we conduct applied population health research and disseminate and evaluate community health interventions, including our Healthy Communities Initiative designed to reducing health disparities in our region.

Our team (of mostly nurses and health educators) was doing amazing work in our communities, but struggled to prioritize these efforts and evaluate their effectiveness. While on a conference call for the health disparities workgroup, one member offered that we were providing an intervention targeted for low-income, Spanish-speaking populations in a wealthy suburb nearby. After I picked myself up off the floor, I asked the obvious question: “Why are we dedicating our limited resources to address health disparities where they do not exist?” Unfortunately, the answer was: “Because they asked us to”.

This exemplified the need to move our team toward data-based decision-making to guide program dissemination and evaluation metrics, but to do so in an accessible manner that built upon existing strengths and history.

Hot Tip: To move beyond jargon, we utilized GIS mapping of our community on key socioeconomic and demographic indicators alongside hospital utilization data to open up a discussion that helped our staff move from defensiveness over not responding to community requests to prioritization toward areas of greater need. By presenting maps accompanied by data tables, we matched the “hard data” with their lived experience working in the laundromats, grocery stores, and Hindu temples in those areas. “I know that neighborhood” was the rallying cry, creating space to build on their expertise and identify intervention opportunities in target communities.  

Lessons Learned: Our staff initially reacted to the word “evaluation” as if it were Ebola. They expressed unexpected confusion and concern and felt threatened. We discovered that it was the semantics, not the practice, that put up a barrier. On their own, staff developed elaborate systems for tracking repeat participants in community-based events. They wanted to know if their efforts were making an impact. By building on this desire and re-framing the question, we engaged them in a process to redesign the Initiative, eventually adding place-based approaches to our direct outreach.

Rad Resource: The Vulnerable Populations Footprint from Community Commons was one tool that we used to quickly visualize our communities and guide the conversation. By giving staff the ability to “play around” with the map, you can open up new lines of inquiry and give them buy-in on the resulting decisions.

The American Evaluation Association is celebrating CP TIG Week with our colleagues in the Community Psychology Topical Interest Group. The contributions all week come from CP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

My name is Kyrah Brown. I am a community psychologist and the program evaluator at Ubuntu Village, a grassroots African-centered community education program in Kansas. I would like to share some lessons from my experience working with an African-centered program that is addressing health disparities through community education and grassroots collaboration.

Ubuntu Village operates as both a Saturday school program and a hub by which collaborative partnerships can be formed to address relevant community issues. Following an initial needs assessment, Ubuntu Village will be partnering with other groups to provide a series of workshops focused on health education. It is important to note that Ubuntu Village is nested within a community that, for years, has been the focus of various interventions aimed at reducing health disparities (e.g. infant mortality and diabetes). This can be problematic particularly when such efforts fail to recognize or understand the cultural and community context. Grassroots programs such as Ubuntu Village are needed because they play a critical role in filling gaps that may exist within community.  One of my primary tasks has been to build an infrastructure for evaluation within the program and in doing so, I have gathered new insight into effective and responsive programming.

Hot Tip:  Seek out others who have experience or expertise in designing and evaluating programs specific to the culture you are working with. I found it extremely helpful and reassuring to reach out to various individuals who worked with African-centered programs. With many different types of African-centered programs, it really does help to examine and learn from the various evaluation approaches that are being used.

Lesson Learned: You must be prepared to be adaptive and creative. The process of building an infrastructure for evaluation at Ubuntu Village, although rewarding, has been challenging. I found it difficult to find information about African-centered evaluation models (outside of education programming) from which I could pattern my efforts. In the end I found it necessary to trust in my knowledge base and use some creativity in building an appropriate evaluation approach. 

Hot Tip: Using a culture-centered lens, in my case an African-centered lens, in every phase of the evaluation is valuable. Health disparity interventions are not one-size-fit-all, even if the target populations appear the same on the surface.  Failing to adjust your lens ignores the program context which, in this case, is essentially rooted in rich cultural and community values.  In my experience, using this lens helped better identify the needs and resources of the community.

The American Evaluation Association is celebrating CP TIG Week with our colleagues in the Community Psychology Topical Interest Group. The contributions all week come from CP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hello!  I am Julianne Manchester of the Community Psychology Topical Interest Group.   In this discussion of reducing health disparities and achieving health equity, I’d like to focus on organizational readiness and practice as a starting point for delivering effective programs, products, and services within the context of improving healthcare access, raising standards of cultural and linguistic competency of providers, or improving community-based evaluation approaches.  

Lesson Learned: It is not enough to have an organizational mission or strategic endeavor to reduce health disparities.  The climate of the organization itself can further the effort through prioritizing discussion of health disparities on meeting agendas, regularly revisiting the sector makeup of stakeholder groups involved in organizational planning (boards, committees, coalitions), and including youth in planning outreach efforts.

The roots of change begin in the local communities, with local conversations.  The Dayton, Ohio Local Office of Minority Health (LOMH), located at Public Health – Dayton & Montgomery County (PHDMC), has adapted the National Stakeholder Strategies for Achieving Health Equity, National Partnership for Action to End Health Disparities (full narrative outlining five goals available online) into a checklist for completion at the organizational level. It was developed through key informant interviews with community leaders and piloted on-line in the spring of 2013.

The NPA strategies prioritize five domains of organizational activity:

  1. Public Awareness;
  2. Youth Involvement;
  3. Healthcare Access;
  4. Cultural and Linguistic Competency
  5. Data, Evaluation, and Research.

Rad Resource: The items on the Dayton LOMH-created questionnaire provide some operational definitions and metrics by which organizations can monitor their activities on the overarching goal statements.  The tool is described in the Fall, 2014 edition of The National Partnership for Action (NPA) Quarterly Newsletter.

Rad Resource: The PHDMC developed the NPA Checklist, available for organizations to assess their strengths in the five domains of organizational activity described above.

The American Evaluation Association is celebrating CP TIG Week with our colleagues in the Community Psychology Topical Interest Group. The contributions all week come from CP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

<< Latest posts

Older posts >>

Archives

To top