AEA365 | A Tip-a-Day by and for Evaluators

TAG | organizational culture

Hello from D.C.! My name is Kristin Mendoza, an alumna of the AEA Graduate Education Diversity Internship (GEDI) program 2014-15 cohort. I am a recent MPH graduate from George Washington University and I am excited to share with you my experience as a GEDI at the Office of Science Planning and Assessment (OSPA) at the National Cancer Institute. OSPA serves as evaluation, assessment and strategic planning consultants across the institute.

As a GEDI scholar I worked on the Program Assessment Branch on a number of projects. I also had the unique opportunity to work at NCI with another GEDI. It was an incredibly enriching experience and the training provided by AEA continues to drive my professional development today.

Helpful Hint #1: Always, always engage your stakeholders. In some cases, the evaluator’s role also includes sitting down with your stakeholders/clients and identifying the priority focus areas. Evaluators will be constrained by budget, resources, time, etc. and it is important to maintain the quality of the work as best as they can. Communicating with your stakeholders will help mitigate any set backs due to resource constraints.

Helpful Hint #2: Culturally responsive evaluation is defined in many ways and it is important to assess how your work environment, office or organization practices it.

Helpful Hint #3: Flexibility is key. Conducting an evaluation may introduce desired or undesirable results/findings. It is important for the work plan to be flexible, to communicate with the client when things come up and to identify ways to further explore surprising results/findings.

Rad Resource for New Evaluators: The evaluation theory tree originally presented by Dr. Christina Christie and Dr. Marvin Alkin currently at the University of California, Los Angeles. There are many versions of the tree that have come after their original publication. This resource helps new evaluators understand and visually see the different evaluation theories in existence as well as the practitioners. It definitely helped put things in perspective for me.

The American Evaluation Association is celebrating Graduate Education Diversity Internship (GEDI) Program week. The contributions all this week to aea365 come from AEA’s GEDI Program and its interns. For more information on GEDI, see their webpage here: http://www.eval.org/GEDI Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

We are Rebecca Stewart, Chief Practice Officer, and Samantha Hagel, Chief Administrative Officer, both with The Improve Group, a firm based in St. Paul, Minnesota, with a mission to help mission-driven organizations make the most of information, navigate complexity, and ensure their investments of time and money lead to meaningful, sustained impact.

This year, we decided to consider a beautiful question: What is The Improve Group’s role in creating a diverse, inclusive field of evaluation? The question emerged as we were thinking about the 2015 Year of Evaluation and its focus on equity, on the AEA’s cultural competency statement, and our own desire to promote social justice. As we’ve pondered this question, we realize part of our role is to share lessons learned and success stories with others in the field of evaluation.  So, here goes!

Hot tip: One way we support diversity and inclusion in our practice is to utilize a competency model. Our competency model asks all of our staff to have a constant awareness of, be learning about, and apply cultural competence. It is not a one-time thing; we want to see team members thinking about this all the time, unprompted. We support them with several opportunities to reflect and learn from each other, including an organization-wide conversation on unconscious bias.

Lesson Learned: Develop strategies to diversify the pipeline of people entering the field of evaluation and applying for open positions. Over the years, the vast majority of our candidates have come from a single graduate program. This year, we are experimenting with connecting with non-traditional audiences. For example, we gave presentations in undergraduate programs, and to Vista and AmeriCorps members at Public Allies and the PSEI Vista Program, to raise awareness of evaluation as a profession. We also changed our internship program from a purely graduate-level program to a summer internship for undergraduates and a school-year partnership with the GEDI program.

Hot tip: Be expansive, curious, and collaborative in seeking out community partners. We got to know an organization, Partnership Resources, through an evaluation of how the Americans with Disabilities Act has affected employment for people with disabilities. This interaction challenged us to figure out how to be a supported employment site. We looked at our workplace in a new way and found a role for a new employee, matched to us by Partnership Resources.

Rad resource: We partnered with New Sector Alliance for our new summer internship program. We have a shared interest in broadening the scope of those entering the social sector, and they have helped us reach potential new evaluation professionals.

Interested in further conversation? Join us at the conference: http://bit.ly/1PbkqOV

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Greetings aea365 community! I’m Ann Emery and I’ve been both an external evaluator and an internal evaluator. Today I’d like to share a few of the reasons why I absolutely love internal evaluation.

Lessons Learned: Internal evaluation is a great career option for fans of utilization-focused evaluation. It gives me opportunities to:

  • Meet regularly with Chief Operating Officers and Executive Directors, so evaluation results get put into action after weekly staff meetings instead of after annual reports.
  • Participate on strategic planning committees, where I can make sure that evaluation results get used for long-term planning.

Lessons Learned: Internal evaluators often have an intimate understanding of organizational history, which allows us to:

  • Build an organizational culture of learning where staff is committed to making data-driven decisions.
  • Create a casual, non-threatening atmosphere by simply walking down the hallway to chat face-to-face with our “clients.” I hold my best client meetings in the hallways and in the mailroom.
  • Use our organizational knowledge to plan feasible evaluations that take into account inevitable staff turnover.
  • Tailor dissemination formats to user preferences, like dashboards for one manager and oral presentations for another.
  • Participate in annual retreats and weekly meetings. Data’s always on the agenda.

Lessons Learned: Internal evaluators can build evaluation capacity within their organizations in various ways:

  • I’ve co-taught Excel certification courses to non-evaluators. Spreadsheet skills can help non-evaluators feel more comfortable with evaluation because it takes some of the mystery out of data analysis.
  • I’ve also led brown bags about everything from logic models to research design. As a result, I’ve been more of a data “coach,” guiding staff through evaluation rather than making decisions on their behalf.

Hot Tips: Internal evaluators can use their skills to help their organizations in other ways, including:

  • Volunteering at program events. When I served food to child and teen participants at Thanksgiving, my time spent chatting with them helped me design more responsive data collection instruments.
  • Contributing to organization-wide research projects, such as looking for patterns in data across the participants that programs serve each year.
  • Partnering with graduate interns and external evaluators to conduct more in-depth research on key aspects of the organization.

Cool Trick: Eun Kyeng Baek and SeriaShia Chatters wrote about the Risks in Internal Evaluation. When internal evaluators get wrapped inside internal politics, we can partner with external evaluators like consulting firms, independent consultants, and even graduate interns. Outsider perspectives are valuable and keep things transparent.

Rad Resources:

AEA is celebrating Internal Evaluators TIG Week. The contributions all week come from IE members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

· · · · · · · · ·

I’m Norma Martinez-Rubin, an independent evaluation consultant and Independent Consulting TIG Chairperson 2011. Here I present a few lessons learned in transition from being an external to internal consultant.

Claiming to be multicultural boosts the self-confidence required upon beginning a new evaluation project. Combine that with visualization of near success and that increases my confidence by volumes. Whether I use such “tricks” to approach a short-term or multi-year project, my recurrent challenge is to find the right mix of approachability, credibility, contribution, and professional satisfaction to come away from the project with a sense of integrity and professionalism. Arming oneself with professional standards and competencies is a requisite, but that doesn’t necessarily minimize what I call the “people factor” —the intricacies and idiosyncrasies within human relations that undoubtedly influence one’s performance as an evaluator in organizational cultures with their own vocabularies, operational flows, and personalities.

Picture yourself as the evaluator on a project requiring your expertise, flexibility, patience, and skills to discern pertinent issues from personal preferences of project stakeholders surrounding you. What are the agendas and preferences at play?

 

Lessons Learned

Understanding one’s emotions and their management extends beyond the field of psychology into organizational and professional development. Daniel Goleman (http://danielgoleman.info/topics/emotional-intelligence/) expanded the concept of emotional intelligence from it origins within the field of psychology. Since the mid-90s, his writings have been used for leadership development and business development courses. And for good reason. Personal and social competence combined with intellectual competence makes for a better-rounded individual. Our world could stand to have more people who reflect upon that.

Evaluators have recognized the quality of being reflective among the necessary elements that distinguish quality evaluation. Based on a review of manuscripts submitted for publication in a forthcoming book on qualitative inquiry, Janet Usinger at the University of Nevada, Reno and colleagues identified that among the Three Hallmarks of Quality in qualitative evaluation. The other two were transparency and an educative process (Evaluation 2010, Session #314, “Quality In Evaluation: How Do We know It When We See It in Qualitative Evaluations?”)

Hot Tip: Participation in a local chapter of a management consulting organization (www.imcusa.org) exposes one to varied disciplines outside of the field of evaluation. In a collegial and professional atmosphere, learning from management consultants outside of the evaluation field increases one’s appreciation of those “external factors” typically considered when designing a logic model or presenting alternative explanations for evaluation findings. In other words, one’s worldview is expanded. That’s a good thing that transfers to better communication with prospective clients and/or program stakeholders no matter the culture.

The American Evaluation Association is celebrating Independent Consultants (IC) TIG Week with our colleagues in the IC AEA Topical Interest Group. The contributions all this week to aea365 come from our IC  TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · · ·

Archives

To top