AEA365 | A Tip-a-Day by and for Evaluators

TAG | mentoring

My name is Christina Peterson. Shortly after starting my first semester as a PhD student at the University of Tennessee, I had an opportunity to participate in an evaluation project as a data collector. It was just the type of experience I felt I needed to establish myself as an emerging evaluator in my community. Instead, it ended up being a lesson on establishing my ethical boundaries. Fortunately, the faculty in my department had provided the Rad Resources I needed to navigate this situation early in my PhD career.

Rad Resources:

Researcher Journal. Reflexivity is a critical evaluator competency. During my first semester, we were required to start a reflexivity journal to explore our growth as evaluators. The first exercise was to write about our personal ethics in conducting evaluation. I found clarity in the key words from this statement: self-determination, transparency, and autonomy. I described how it was important to me that people were respected and had a voice in the evaluation protocol. Because of the journal, I was also able to reflect back on the two major themes that emerged from my entries about this situation: integrity and social justice.

AEA Guiding Principles. The concerns I saw emerging in my journal are addressed explicitly in the Guiding Principles of the American Evaluation Association. Two principles provided the direction I needed to make a decision about moving forward with the data collection: integrity/honesty and respect for people. One of my concerns about the project was that the poorly constructed survey items would provide misleading information about the population. Furthermore, since there was no clear purpose for the data collection and the survey protocol did not include informed consent, I was not confident that this work would maximize benefit and reduce unnecessary harm to the community.

Mentorship. Although I felt confident that I needed to let go of this opportunity, there was a lingering feeling of self-doubt. Who was I to question the survey protocol of a professor? What did I know about conducting field research? The AEA Guiding Principles provided direction, but is that how evaluation really works in practice? For the answer, I turned to a faculty member who I knew had talked openly in class about making similar ethical choices as a novice researcher. She reassured me that this protocol was not business as usual in field research and we discussed the courses of action I could consider.

Later that evening, I notified the lead researcher that I could not continue my participation in the data collection and, in line with the AEA Guiding Principles and my personal ethics, I was transparent about my concerns. These Rad Resources were essential to my development as an ethical evaluator.

reflective pool of water in brick pavers

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

My name is Art Hernandez, Visiting Professor at the University of the Incarnate Word in San Antonio, Texas.

I was an AEA MSI Fellow early on and have served as the Director for several cohorts – most recently this past year. I serve and have served as evaluator and teacher of evaluation and am very interested in the processes of cultural responsiveness in practice especially in regard to measurement and assessment. As one of the early AEA members from a Latino background, I have been positioned to offer my perspective to others on all aspects of Evaluation and professional practice.

Lesson Learned:

In my case, the relationships I have established and from which I have derived or offered insight, etc., resulted from “natural relationships” formed by my association with those Latinas/os who originally sought me out.  It is also clear to me that whatever benefit may have resulted to others, I certainly benefited.

It was these naturally occurring relationships which provided the means to advance the cause of representation and leadership for Latinas/os.  Clearly, because these relationships were organic, there was time to develop a foundation of trust- that any implied commitment of support could be trusted, that the motivation to be of support, to advance and advocate was genuine and time to develop a means of communication which reflected shared values predicated on a mutual desire to continue the relationship (friendship) for the long term.

Hot Tips: 

  • Interested and invested individuals and groups can make a difference in defining and shaping the “success” of our peers.
  • For underrepresented individuals, it is important to seek out interested, invested others even if they provide no more than social support and evidence that full participation is possible.
  • For those already within the “system” it is important to remember that even if you take no deliberate action, your presence and attitude toward newcomers conveys a great deal about the nature of the organization and the likelihood of success.
  • Numbers matter. Increasing the representativeness of constituent groups so that their “voice” can exercise influence should be a priority.
  • Diversity is of value to organizations which can benefit from a greater reach, improved retention and performance, increased innovation, social relevance and improved morale and sense of safety for those from underrepresented groups.
  • Informal mentoring is as valuable as formal mentoring.
  • Social and professional networks are important contributors to individual and organizational success.
  • Mentors, especially those from the majority who serve protégés from minority cultural backgrounds, should be sensitive to comments and attitudes of others and seek to advocate and advantage their protégés in the face of suspected prejudice or bias.
  • Mentors should be prepared to learn as well as to teach.
  • Mentors should expect, encourage and support protégés to achieve success – even that surpassing their own.
  • Finally, every field of endeavor benefits from efforts to embrace and exercise cultural responsive practice. The success of these efforts will be determined in no small way by the inclusion of experts who have firsthand, natural experience and knowledge of other cultural identities.

Rad Resource:

Norman, R.L. (2011). Five Best Practices for Cross-Cultural Mentoring in Organizations

The American Evaluation Association is celebrating Latina/o Responsive Evaluation Discourse TIG Week. The contributions all this week to aea365 come from LA RED Topical Interest Group members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

We are Jennifer Johnson and Melanie Meyer, and we are evaluators at the Florida Legislature’s Office of Program Policy Analysis and Government Accountability.  Jennifer has served as Past President and currently serves as Secretary for the Southeast Evaluation Association (SEA), and Melanie holds a Master’s degree in Adult and Continuing Education and is a member of SEA.

The purpose of continued professional development is to help individuals maintain competence in their profession.  It is critical to organizational success and ensures individuals possess current knowledge and skills to effectively do their jobs and contribute to organizational goals.

Cool Tricks: Two ways organizations can address continued professional development are through mentoring and training.  Mentoring consists of a one-on-one relationship in which an individual serves as a mentor or coach to an individual less experienced or new to the organization.  A training program occurs in a group setting in which one or several individuals adopt a trainer/teacher role.

Hot Tips: Developing mentoring and training programs requires significant organizational commitment.  Below are a few tips organizations should consider:

Mentoring Programs

  • Build trust. Ensure the mentor relationship is only about support and growth, not supervision or management.  Mentors should not provide performance evaluations.
  • Specify the mentoring role. Determine what areas mentors should address, e.g., specific skills and knowledge, or organizational processes and culture.
  • Clarify the frequency and method of contact.  Determine the length and format of interactions (e.g., face-to-face or phone; structured meetings or less formal interactions) and whether mentors should be available between sessions.
  • Determine the duration of relationship.  Determine whether the mentoring relationship should be a time limited or ongoing. Periodically assess the relationship to ensure both individuals are happy with the process.

Training Programs

  • Assess internal expertise.  Identify individuals in your organization to provide training; it is the most effective way to develop and sustain a training program and is inexpensive, flexible, and allows the trainer to be available for post-training consultation.
  • Vary the scope and format.  Provide focused training for essential skills and broader training for areas that require a general knowledge base.  Use a range of formats including classroom-style presentation, interactive and hands-on sessions, and one-on-one tutorials. Develop self-guided materials.
  • Tailor programs to skill levels Specialize training for beginner, moderate, and advanced skill levels.
  • Encourage participation by management.  Demonstrate to the organization that training is important by encouraging managers to attend and look for opportunities for individuals to apply what they learn.

The American Evaluation Association is celebrating Southeast Evaluation Association (SEA) Affiliate Week with our colleagues in the SEA Affiliate. The contributions all this week to aea365 come from SEA Affiliate members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Hi! This is Nivedita Ranade and Tom McKlin with The Findings Group.  We are evaluating two federal-level grants that provide mentoring to students with disabilities.  These mentoring relationships are critical to program success because the students see the mentor as their primary source of support.  The program administrators qualified these relationships as “nurturing” because the mentor is invested in the students’ maturation and psychosocial development.

Lessons Learned:

  • The quality of nurturing makes the role of the mentor unique and different from other advisers.  An academic adviser is concerned with a student’s course load and grades.  An Office of Disability Services advisor is concerned with providing the right services and accommodations.  However, a mentor is concerned with the overall success of a student.  Rather than targeting just one aspect of the student’s life (e.g. academic), the mentor takes a holistic approach towards the students’ needs.
  •  The mentor is a nurturer because he/she responds to students’ needs and trajectories.  A freshman student who is struggling to adjust to campus life may need assistance in terms of stress and time management as opposed to finding an internship.  On the other hand, a senior student who is hoping to move on to a job after he/she graduates may benefit from having an internship to build his resume and professional skills.
  • Nurturing corresponds to scaffolding. This idea is based on Vygotsky’s Social Development Theory, which asserts that learning typically depends on interactions with a more knowledgeable/competent other.  Knowledge is acquired as novices interact with experts and peers and engage in “legitimate peripheral participation” wherein the novices observe the practice of experts and become experts over time.  Learning in these contexts is generally scaffolded to provide supports until novices move from peripheral to full participation.
  • Nurturing leads to self-advocacy.  Self-advocacy leads to self-determination and personal responsibility.   By helping the students to self-advocate, the mentor helps the students re-work their conceptions of what it means to have a disability.  Students learn to define and come to terms with their disability, which in turn enables them to disclose their disability in situations where it is necessary to do so.  The confidence that comes with self-advocacy leads to increased personal responsibility and self-determination because students feel that they can achieve what they want despite their disability.
  • Thus, nurturing is an essential quality in a good mentor, and we surmise that the field might benefit from a survey construct that focuses on nurturing in mentor/mentee relationships.

 Rad Resource:

  • How to measure mentoring: We have compiled a mentor-mentee survey based on these lessons learned.  Please contact us at nivedita@thefindingsgroup.com.

The American Evaluation Association is celebrating the Disabilities and Other Vulnerable Populations TIG (DOVP) Week. The contributions all week come from DOVP members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

Greetings, I am Chelsea Heaven, a recently graduated master’s level biostatistics and public health student. As with many of my fellow recent graduates, I am always on the lookout for as many opportunities as possible to kick-start my young career in statistics and evaluation. A colleague of mine who I work with at my campus’ statistical consulting center recommended becoming involved with Statistics Without Borders (SWB), and I could not be more grateful to them.

My experience with SWB has been invaluable for my professional development. SWB is an all-volunteer organization run as part of the American Statistical Association. Involvement in SWB has exposed me to real data that is used in the daily operation of health care facilities in East Africa, and given me the opportunity to formulate recommendation reports to NGO’s who would use this data for health policy decisions.

The intention for me writing this blog post is reach out to other graduate students involved in evaluation and share how involvement with SWB could be beneficial for their evaluation portfolios, and prepare them for “the real world” of program evaluation.

Lessons Learned:

  • It’s important to gain experience with real-world data.Gain experience with the real-world, messy data that is all too often present in evaluation projects.
    • Real data (i.e. data that does not come from textbooks) is often messy, comes in many different formats, and requires substantial effort and teamwork before it can be interpreted. Working with SWB projects allows one to have direct experience with messy data in a team environment, which is invaluable preparation for evaluation projects that happen in “the real world”.
  • It’s important to network.  Network and collaborate with evaluation professionals from around the world.
    • Collaboration is the name of the game nowadays in research and evaluation projects, and oftentimes these collaborations happen over e-mail and conference calls. When you are placed in a project at SWB, you are put on a team with 4-5 diverse professionals who you must effectively coordinate with to produce high-quality work. Therefore, SWB gives students highly beneficial experience working in a long-distance collaborative team environment – which will prepare them for future collaborations in their professional careers.

Hot Tip:

Rad Resources:

  • Join Idealist. SWB is a member organization of Idealist. Consider joining Idealist, and then joining the SWB site.
  • Join SWB. Anybody interested applying statistical and evaluation knowledge to the international community is welcome to volunteer at SWB.
  • Volunteer as a web-based statistical mentor for students in a developing country.
  • Volunteer at Peoples-uni. Courses of particular interest would be Evaluation of Interventions, Biostatistics, Public Health Concepts for Policy Makers, and Health Economics all with evaluation and research components.

· · ·

My name is Norma Martinez-Rubin and I am an independent program evaluator, occasional trainer, and public health practitioner. Having chosen program evaluation as a second-career focus (www.EvaluationFocused.com) the necessity to ramp up, jump in, and practice the art and science of evaluation as best possible led me to revisit a career strategy to which I was introduced a few decades ago: build your skills by working with a preceptor or mentor. But how does one find that mentor? Where does one start? How do we enlist their participation in our development? Since I joined AEA in 2006, I’ve explored some of the AEA’s many offerings, actively participated, and remained involved to varying degrees. I’ve come to realize that one doesn’t necessarily need a formal arrangement to be mentored. The “right” mentor will be the person who willfully takes a few minutes to guide you to a resource, listens to you, or simply exchanges a thing or two that they’ve learned and thus, frees you to take that (or not) and make it your own. You know that query, “What would you do if you couldn’t fail?” It’s a reminder of the requisite optimism and willingness to test, try, and move on for more —-successful or not, there’s always more!

I realize that there are readers of this blog who are interested in a mentor-mentee arrangement that is more formal than that. Becky Melzer (www.evaluationedge.com) and I reported a bit on that in 2009. (See our posts in the AEA e-Library.) In the absence of any formal mentor-mentee arrangement, here are some tips to get you in a frame of mind to seek out colleagues with whom you can exchange pearls of wisdom:

Hot Tip: Mix business with pleasure . . . Attend a TIG’s business meeting while at the next AEA conference. You’ll get a feel for the group’s sense of collegiality, identify with who you’ll remain in contact through the year, and determine how you might contribute. In a volunteer-run structure, seldom is a person’s time and energy turned away. What fuels a potential mentor-mentee relationship, however subtle, is a continual interest in finding ways for mutual benefit.

Hot Tip: S-T-R-E-T-C-H Consider initiating a mini-project with others. For example, seek assistance to jointly present or contribute toward an AEA conference presentation. Among the benefits of working collaboratively are the small lessons (e.g., meeting timelines, negotiating, asking for help) you learn about yourself in relation to another.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

I’m Tamara Bertrand Jones and I evaluate programs and services in Student Affairs at Florida State University. I’ve always been intrigued by the ways that different people develop the skills they need in order to be successful in their chosen profession. Fundamental to the success of budding evaluators is the important interaction with successful evaluation role models.

AEA has many programs that promote the exchange of knowledge between novice evaluators and more experienced professionals, including the Pipeline Program, the Graduate Education Diversity Internship Program, and the Multi-Ethnic Issues in Evaluation TIG sponsored mentor lunch held during the annual conference. The programs provide formal ways to link mentors and mentees.

Developing mentoring relationships with other evaluators not only helps to provide the evaluation exposure needed, but can also be a source of personal and professional support for both the mentor and the mentee. For graduate students or new evaluators not already engaged in a mentoring relationship, here are some tips that to help you take the first step in developing a mutually beneficial relationship.

Hot Tip: Develop a list of characteristics you desire in a mentor. Do you want someone who has the practical evaluation skills, communication skills, and/or professional standing you can aspire to?

Hot Tip: Seek mentors through relationships developed by attending professional association meetings and conferences, professional development workshops, and other gatherings where evaluation is a focus. These connections serve as a means to hear about opportunities in the field. In addition, these networks facilitate sharing, collaboration, professional visibility, and skill development.

Hot Tip: After you have developed your list of characteristics and have identified potential mentors, reach out to them. Send an introductory email; schedule a meeting to establish a personal connection. At the meeting, discuss your future goals and career plans, and how you envision the relationship benefitting both you and the mentor. Remember that mentoring relationships work both ways.

Hot Tip: Be clear about expectations of your mentor. Communicate to confirm that his/her expectations align with yours. Do not expect one mentor to meet all of your needs. If, after you have discussed expectations and one mentor cannot provide everything on the list, continue to pursue the relationship with your newly revised expectations. Take the time to find additional mentors that can meet your remaining needs.

Hot Tip: Be open to mentors from a different gender or ethnicity. Just as one mentor cannot meet all of your needs, mentors of different ethnic backgrounds or genders bring different aspects to the relationship based on their lived experiences. These experiences may be different from your own and can add a depth of perspective that you may not have previously considered. This heightened awareness can only serve to improve your evaluation skills.

·

Archives

To top