AEA365 | A Tip-a-Day by and for Evaluators

TAG | cultural evaluation group

Hello, we’re Dominica McBride and Leah Neubauer, members of the AEA Public Statement on Cultural Competence Dissemination Working Group. At AEA 2012, we participated in the brownbag session Critically Reflecting and Thinking about Cultural Competence in Your Evaluation Practice. We’d like to share some of the great insights generated from this session.

In this session, we learned about a nursing student who, influenced by Western Medicine cultural ideologies of time- and task-orientation and protocol, misdiagnosed a patient due to a lack of cultural sensitivity and critical self-awareness. We also learned about a comprehensive model of self-reflection from Hazel Symonette that encompasses internal, external/other, space, and time. Below are several lessons learned:

Lessons Learned

Our sense of time can affect how we do our evaluations and the use and impact of our evaluations.In the case study of the nursing student, she quickly ran through her nursing duties, completing an assessment and diagnosis in under 5 minutes. This rush caused a missed opportunity to gain more insight and the patient was misdiagnosed. We, as evaluators, can do this same thing. If operating in a mental frame of “getting stuff done” and “getting it done fast”, we miss opportunities to collect valuable data and gain additional insight. We also miss the opportunity to be more self-reflective. Without this introspection, we may, like the nursing student, “misdiagnose” or misinterpret the situation and data.

We too often assume that we know.Even in the midst of data collection and trying to be objective, we can often assume that we know something or have an accurate interpretation when we actually do not. In Love’s Executioner, Irvin Yolam, a reputable psychiatrist, talks about how communication is filtered through our experiences, biases, feelings, etc. We never fully accurately know what the other is communicating. However, with self-reflection and dialogue, we can check to see if we’re interpreting correctly. We create spaces to gain other perspectives accurately, thus enriching our interpretations and leading to accurate and comprehensive work.

We strive for perfection – is perfection possible?  In Becoming a Critically Reflective Teacher, Dr. Stephen D. Brookfield discusses teaching culture that strives for the ‘perfect 10’ in their teaching. A critically-reflective state of practice acknowledges a constant state of knowing and learning more.

One lesson harkens back to a lesson from SpidermanWith great power comes great responsibility. As evaluators, our judgments are seen as mattering more than some others. We have a special responsibility to ensure that findings and conclusions are inclusive, public, and shared.

Rad Resources

Hazel Symonette developed a comprehensive model for self-reflection we can easily incorporate into evaluation and daily life.

Stephen D. Brookfield has written extensively on the role of critical self-reflection for adult educators.

This week, we’re diving into issues of Cultural Competence in Evaluation with AEA’s Statement on Cultural Competence in Evaluation Dissemination Working Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

I am Jenny Jones, member of AEA’s Public Statement on Cultural Competence in Evaluation (the Statement) Dissemination Working Group. Over the past five years, AEA has honored Asa G. Hilliard’s legacy through an annual Think Tank. The Fifth Annual Asa G. Hilliard III Think Tank on Relational Ecosystems explored how professional roles and relationships can impact the evaluation process and served as the inspiration for this aea365.

Lesson Learned: Dr. Hilliard’s work parallels many of the values highlighted in the Statement. Some essential components of the Statement on Cultural Competence that are aligned with Hilliard’s perspective are the acknowledgement of the complexity of cultural identity; recognition of the dynamics of power; aiming to eliminate bias in language; and, employing culturally appropriate methods.

Hot Tip: Embrace complexity. Some things are best understood by examining the whole and any relationships rather than by attempting to simplify by sorting, piecing-out or dividing into separate parts. The Think Tank introduced eight cultural precepts that we can consider in the scope of assessment and evaluation in a complex ecology. The cultural precepts are Consubstantiation, Interdependence, Egalitarianism, Collectivism, Transformation, Cooperation, Humanism, and Synergism. These cultural precepts serve to improve a standard of action or conduct consistent with cultural competence in evaluation. Below are examples of how the precepts interdependence, egalitarianism, and collectivism are operationalized in evaluation practice.

Interdependence: Asserts that everything in the universe is connected. The ontological origins are relational—everything is inextricably interdependent. For example, when conducting an evaluation one must take a holistic view or sense of interrelatedness of the target population, understanding that a client is connected to a community, a family (biological or constructed) and an agency.

Egalitarianism: Asserts the correct relationship between people is one of harmony and balance. For example, when using this concept in evaluation one must take a non-hierarchical epistemology (what is worth knowing) approach, meaning one must take an approach that does not posit objectivism over subjectivism or Western epistemologies over indigenous epistemologies. Rather all ways of knowing are equal.

Collectivism: Assert that individual effort is a reflection and/or instrument of communal or collective concern. For example, the decision made by an individual regarding who is considered the stakeholder in an evaluation affects the entire community not just that individual. There’s a sense of collective responsibility; nothing one person decides impacts them solely.

Rad Resource: For more information about the life and legacy of Dr. Asa G. Hilliard visit Georgia State University’s Alonzo A. Crime Center for Urban Educational Excellence.

This week, we’re diving into issues of Cultural Competence in Evaluation with AEA’s Statement on Cultural Competence in Evaluation Dissemination Working Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

I am Melanie Hwalek, CEO of SPEC Associates and a member of AEA’s Cultural Competence Statement Dissemination Core Workgroup. My focus within the Workgroup is to help identify ways to disseminate the Statement and integrate its contents into evaluation policy. AEA’s Think Tank: Adoption of the AEA Public Statement on Cultural Competence in Evaluation: Moving From Policy to Practice and Practice to Policy gave me three big ideas for doing this.

Lesson Learned: Cultural Competence can be in big “P” policy and small “p” policy. Dissemination of the Cultural Competency Statement doesn’t have to start with federal or state level, big “P” policy change. Small polices like setting criteria for acceptable evaluation plans, for assuring that evaluation methods take culture into consideration, and for ensuring culturally sensitive evaluation products can go just as far – or further – in assuring that all evaluations validate the importance of culture in their design, analysis, interpretation and reporting.

Hot Tip: Start where there is a path of least resistance.Agencies that exist to represent or protect minority interests are, themselves, culturally sensitive. These are the agencies that should easily understand the importance of assuring that the evaluations of their programs should include cultural competence. If you are passionate about infusing cultural competence into municipal, state or federal policy, start with these types of agencies since they are likely to understand the importance of culturally sensitive evaluations. Keep in mind, though, that just because an organization “says” it values cultural competence doesn’t mean the really know how to be and act in a culturally competent way.

Hot Tip: Try to go viral.Infusing cultural competence into policy means that we need to be open to all kinds and levels of policy, much of which is identified only through practice. The lesson here is to start promoting cultural competence to anyone and anywhere evaluation planning, methods, analysis and reporting are discussed. In this networked world, the more people who think and talk about cultural competence in evaluation, the more likely it will find its way into evaluation practice and evaluation policy.

Rad resource: William Trochim wrote an informative article on evaluation policy and practice.

This week, we’re diving into issues of Cultural Competence in Evaluation with AEA’s Statement on Cultural Competence in Evaluation Dissemination Working Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

My name is Cindy Crusto, and I am an associate professor at Yale University School of Medicine and chair of the AEA Public Statement on Cultural Competence in Evaluation Dissemination Working Group. Today, I summarize and reflect on the Hindsight is 20/20: Reflecting on Missed Opportunities, Missteps, and Successes in Attending to Culture and Context in Evaluation Practice Think Tank, an Evaluation 2012 session organized by our working group.

Arthur Hernandez, Kari Greene, and HazelSymonette illustrated missed opportunities and successes in addressing and attending to culture/context in evaluations. This entry focuses on Kari Greene’s fictitious case study with a lesbian, gay, bisexual, transgender, queer (LGBTQ) health coalition conducting a needs assessment. The coalition quickly realized there was not one “LGBTQ community” but instead many different communities. The coalition needed to identify who they wanted to serve –identity-based communities only or include individuals not identifying as LGBTQ but might have LGBTQ health needs (e.g. a man born a woman but transitioned years ago and does not identify as transgender, however, has trans-related health needs)? Would the group address LGBTQ issues statewide, including rural issues and the “gay ghetto” of urban centers? Would they gather information across the lifespan? Finally, given the diversity of individual member’s gender identity, meetings began with individuals indicating their preferred gender pronouns.

Lessons Learned: The case study highlighted several lessons:

  • We have to identify within group diversity
    • cultural similarity of the evaluator and the evaluands does not guarantee awareness of within group diversity;
    • diversity in self-definition/self-identification exists; evaluators may need to bridge generational, regional, and/or other gaps to develop within group understanding, tolerance, acceptance, and/or consensus;
    • We should move away from a cultural competence “checklist mentality” and toward cultural humility, anongoing process of self-reflection and self-critique
      • cultural competence checklists do not exist; each evaluation is different and requires attention to different cultural/contextual issues;
      • yesterday’s culturally competent evaluator is tomorrow’s incompetent evaluator; we must remain organic, dynamic, and ever-morphing, open, responsive, and hold to a reflective beginner stance;
      • We have to attend to issues of power and bias.

The Think Tank’s discussant, Donna Mertens, posed questions that can be used in your work:

  1. What dimensions of diversity are relevant in the communities in which you work?
  2. How do you identify relevant dimensions of diversity in your evaluations?
  3. What power issues might arise between subgroups within the evaluation context? How would you address them?

Rad Resources

This week, we’re diving into issues of Cultural Competence in Evaluation with AEA’s Statement on Cultural Competence in Evaluation Dissemination Working Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi, I’m Dominica McBride, President of The HELP Institute, and member of the AEA Public Statement on Cultural Competence Dissemination Working Group. The Working Group has been hard at work spreading the word about the Statement and its contents. One of the ways we did this was through conference presentations on various areas that the Statement could be applied, including evaluation policy, evaluation practice, and self-reflection. The group coordinated several conference presentations this year that provided insight into how to incorporate cultural competence in our practice, policies, and general thinking. This week, we’d like to share the ideas that came from our presentations:

Cindy Crusto writes on the Think Tank: Hindsight is 20/20: Reflecting on Missed Opportunities, Missteps, and Successes in Attending to Culture and Context in Evaluation Practice. In this think tank, three evaluators came together to share their pitfalls and successes around attending to culture and context in their evaluations. Their experiences ignited an engaging conversation about how we, as evaluators, can further our integration of culture and context in evaluation practice.

Melanie Hwalek writes on the Think Tank: Adoption of the AEA Public Statement on Cultural Competence in Evaluation: Moving From Policy to Practice and Practice to Policy. In this potent presentation, they explored ways in which to integrate cultural competence into policy. Melanie provides some practical tips on moving forward in this area.

Dominica McBride and Leah Neubauer write on the Brown Bag: Critically Reflecting and Thinking about Cultural Competence in Your Evaluation Practice. In this synergistic conversation, the attendees posed critical questions around power, interpretation and misinterpretation, and cultural influences on decision making.

Jenny Jones writes on the Think Tank: Fifth Annual Asa G. Hilliard III Think Tank on Relational Ecosystems, which pays an annual tribute to Dr. Hilliard’s work and influence. From this think tank, Jenny highlights powerful concepts, such as interdependence and egalitarianism, and describes their role and potential influence in evaluation.

Lastly, Karen Anderson shows the significant focus on culture in evaluation by noting the impressive number of sessions that included or touched on culture in this year’s conference.

Hot Tip: Enjoy the week and Happy Holidays!

·

Hi, I’m Dominica McBride, President of The HELP Institute, Inc. and a member of the AEA Public Statement on Cultural Competence in Evaluation Dissemination Working Group. I also have my PhD in Counseling Psychology and have provided psychotherapy. This tip focuses on the affective and psychological side of cultural competence in everyday evaluation practice.

Culturally competent evaluation practice requires self-awareness and self-reflection. So much of our evaluation practice is guided by our decisions. It would be nice to think that our decisions are mostly driven by our frontal lobes – the seat of deliberation and reasoning; however, most of our choices are influenced by our subconscious mind, as discovered through recent neuroscience research. Our subconscious mind (influenced by the limbic part of our brains also known as the emotional brain) is constituted of our experiences, exposures, and emotions. Our experiences literally shape the wiring of our brains and repeated exposures to similar messages connect our brain cells, which leads to more automatic thoughts. So, if someone is exposed to repeated messages depicting Muslims as terrorists, for example, our brains begin to incorporate this. These thoughts become a part of us, even subconsciously, and can negatively affect our interactions and decisions in working with the group, especially in the absence of antithetical experiences. Microaggressions, which are unintentional slights towards a person related to their group affiliation, can begin to develop. They can also show up in interactions and decision making within an evaluation, like forgetting or overlooking the inclusion of a certain group in research or evaluation design. For example, a 21st Century study “found” a lack of facial recognition abilities in African-Americans compared to Euro-Americans. However, due to cultural incompetence, the participants were only shown Caucasian faces. When corrected with cultural competence, there was no difference.

The Statement states “cultural competence is a stance taken toward culture” and “culturally competent evaluators respect the cultures represented in the evaluation.” To be culturally competent and value and respect culture and different communities, the Statement asserts that we must challenge our stereotypes and ameliorate our biases. We have to examine and address the biases hidden in our subconscious that influence our decision making, interactions with others, and evaluation practice.

Hot Tips:

  • Take the Implicit Association Test. This test will inform you of some of your implicit biases.
  •  Examine your biases through journaling and deliberately find and create experiences that counter your stereotypes and make conscious note of experiences that do not support them.

Rad Resources:

  • Blink is a good book that describes our subconscious mind and its influences on decision-making and interactions with others
  • Crash is a provocative movie graphically demonstrating explicit and implicit biases and their effects on others

The American Evaluation Association will be celebrating Cultural Competence Week. The contributions all this week come from the Cultural Competence committee. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

· · ·

Hello, I’m Jori Hall, assistant professor at the University of Georgia and a member of the AEA Public Statement on Cultural Competence in Evaluation Dissemination Working Group. This tip is focused on integrating cultural competence into everyday practice through values-engagement.

Tips:

  • As suggested in the AEA Public Statement on Cultural Competence in Evaluation, all evaluation practice and evaluands are situated in and influenced by cultural norms, values, and various ways of knowing. Values-engagement acknowledges these influences and attempts to be responsive to the dynamic interaction between the values reflected in evaluation practice and the evaluand. That is, values-engaged evaluators understand that evaluation practice promotes values, and that these values must respectfully engage stakeholders’ values.
  • Values-engagement is not a specific strategy or a set of required methods; rather, it is a commitment to culturally responsive evaluation. While there is more than one way to be values-engaged, the commitment to culturally responsive, values-engagement suggested here involves the evaluator prioritizing values of inclusion and equity in everyday practice. Inclusion refers to engaging and describing the plurality of stakeholders’ values, perspectives, and concerns, focusing on the least well served in a particular context. Equity refers to how well and to what extent the evaluand is attending to stakeholder groups (i.e., access, participation, etc.) in the context. Because values-engagement advocates inclusiveness and the equitable treatment of stakeholders, it supports the goals of the Statement on Cultural Competence in Evaluation.
  • Values-engagement can be integrated throughout the life cycle of an evaluation, and enacted through generating evaluation questions, data, and dialogues related to the ways in which the evaluand is attending to the cultural values of the groups represented in the context. To learn more about values-engagement, its connection to cultural competence, and how evaluators can practically enact its commitments in different evaluation contexts, begin with the resources provided below!

Rad resources:

The American Evaluation Association will be celebrating Cultural Competence Week. The contributions all this week come from the Cultural Competence committee. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

· · · · · · · ·

I’m Karen Anderson, AEA’s Diversity Coordinator Intern, and in this role I support AEA’s diversity programs, TIGs, and the AEA Public Statement on Cultural Competence in Evaluation Dissemination Working Group.

The baton has been passed from the Cultural Competence in Evaluation Task Force, the Statement developers, to the Cultural Competence in Evaluation Dissemination Working Group to translate the Statement from paper to practice. One strategy for its broader dissemination and use is integrating the Statement into the policies and procedures of organizations that conduct and commission evaluations.

The AEA Public Statement on Cultural Competence in Evaluation has several core concepts, including the implications culture has for all phases of evaluation, including staffing of evaluation efforts and ensuring that members of the evaluation team collectively demonstrate cultural competence in the context for each evaluation. How does an evaluation practitioner or commissioner begin to do this?

Rad Resource:

Hot Tips:

  • Share the Statement and supplemental resources like Building Culturally Competent Organizations, Key Components to a Culturally Competent System, and It All Starts At The Front Desk with human resources and decision makers in organizations. Recommend the development of a cultural competence committee to monitor and make recommendations for policy revision, program development, and evaluation.
  • Include cultural competence language in the development and response to requests for proposals (RFPs). Check out this post How to Spot a Lip Service Approach to Culturally Responsive Evaluation from Patricia Rogers and Jane Davison’s Genuine Evaluation blog for tips on pointing out when a client may not be walking the walk in relation to culture and program development, theory, and evaluation.
  • If you or other employees at your organization belong to an AEA affiliate, organize an event at your office around the theory or practical applications of the Statement. The Atlanta Area AEA affiliate group hosted one recently, Taking a Stance Toward Culture: Cultural Competence in Evaluation. Reflections from the event can be found in the AEA Newsletter diversity article.
  • Set up a series of lunch and learns to begin having dialogue with colleague to increase awareness and to encourage relationship building, or start a book club discussion using the Statement, and branch out to other reading material to light the spark for cultural competence in evaluation.

The American Evaluation Association will be celebrating Cultural Competence Week. The contributions all this week come from the Cultural Competence committee. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

· · · ·

Hello! We’re Tamera Bertrand Jones, Assistant Professor of Higher Education at Florida State University, Osman Ozturgut, Assistant Professor of Education at the University of the Incarnate World, and Leah Neubauer is the Associate Director of the Master of Public Health Program at DePaul University.

Evaluators enter the field from different academic backgrounds and possess varying levels of professional expertise. Evaluation knowledge can come from academic preparation, on the job training, and professional development.  The AEA Public Statement on Cultural Competence in Evaluation is ripe for use in teaching evaluation. We asked national and international evaluation leaders to provide tips on ways to incorporate the tenets of culturally competent evaluation in teaching evaluation.

Tip #1: Read the statement and dissect it.   Learners can read the Statement and reflect individually and collectively about what the statement means in their specific contexts. Learners could journal their responses to questions like: What are the statement’s strengths and weaknesses? What is missing from the statement? How could the statement be applied in various programs? Instructors could use case studies to pose different scenarios and facilitate a discussion of learners’ various perspectives on solutions relevant to specific contexts.

Tip #2: Determine the learner’s location.  The learner’s location is one’s own cultural position, awareness of others’ positions, and the ability to interact genuinely and respectfully with others. The learner must also assess their technical ability and interpersonal skills.  Once an evaluator is able to adequately assess themselves, they can begin to draw a complete picture of how their abilities may match up different contexts.

Tip #3: Incorporate experiential opportunities.  The old adage “Practice makes perfect” is certainly true when it comes to evaluation.  Evaluation is a hands-on field and the more practice evaluators have in honing their craft, the better equipped they will be for future evaluations.

Tip #4: Encourage the development of mentoring relationships.  One way evaluators improve their competence is by working with other evaluators.  Encouraging learners to build coalitions with other evaluators who are different than themselves, who work in different contexts, or who work with different populations. Others’ personal stories and experiences help us to learn hard lessons without the sting of the actual experience.

RAD Resources:

The American Evaluation Association will be celebrating Cultural Competence Week. The contributions all this week come from the Cultural Competence committee. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

· ·

Hello! We’re Osman Ozturgut, Assistant Professor of Education at the University of the Incarnate World and Tamera Bertrand Jones, Assistant Professor of Higher Education at Florida State University. In the previous posting, we learned about the AEA Public Statement on Cultural Competence in Evaluation (the Statement) and the reason for its development. In this post, we discuss ways to integrate cultural competence into your conference presentations.  The 2012 AEA conference theme, Evaluation in Complex Ecologies: Relationships, Responsibilities, and Relevance, offers a framework to consider the role of culture in evaluation.  Even if your presentation does not directly address culture, there are ways to remain a part of this conversation.  We asked national and international evaluation leaders to provide tips on ways to incorporate the tenets of culturally competent evaluation in your AEA conference presentation to make the list for today.

Hot Tips:

Relationships- Culturally competent evaluators refrain from assuming they fully understand the perspectives of stakeholders whose backgrounds differ from their own. Cultural competence requires awareness of self, reflection on one’s own cultural position, awareness of others’ positions, and the ability to interact genuinely and respectfully with others. Relationships are a foundation from which we build our work. Developing relationships with stakeholders that attend to diverse perspectives ensures different viewpoints are represented. In your presentation, tell the audience how you developed relationships with a variety of stakeholders and included those viewpoints in your evaluation.

ResponsibilitiesEvaluations are not culture free.  Effective and ethical use of evaluation requires respecting different worldviews.Insufficient attention to culture in evaluation may compromise group and individual self-determination, due process, and fair, just, and equitable treatment of all persons and interests. In your presentation, discuss how you did or could have better accounted for culture in your evaluation when you present lessons learned.  How did/will you address power dynamics in evaluations?  What would you do differently, after you’ve read the Statement?

RelevanceThe culturally competent evaluator draws upon a wide range of evaluation theories and methods to design and carry out an evaluation that is optimally matched to the context. Valid inferences require shared understanding within and across cultural contexts.  In your presentation, discuss the context in which your work was located. Describe how evaluation questions were conceptualized and how data were collected and analyzed.  A description of the context and methods provides the audience with a greater understanding of the environment in which your work took place and allows for greater examination of validity.

RAD Resources:

The American Evaluation Association will be celebrating Cultural Competence Week. The contributions all this week come from the Cultural Competence committee. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

· ·

Older posts >>

Archives

To top