AEA365 | A Tip-a-Day by and for Evaluators

TAG | communication

My name is Mimi Doll, the owner of Candeo Consulting, Inc., an independent consulting firm that builds organizations’ capacity to create meaningful change in the communities they serve. Sometimes we can prevent scope creep with good planning, other times no matter how good our preparation is, clients either don’t have a clear sense of what they want or simply change their minds.

Hot Tip:

  • Always Develop a Scope of Services and Contract. Developing a detailed scope of services, including project tasks, work hours, pricing, timeline, roles and responsibilities, makes clear to the client what services and deliverables you plan to provide, and those you don’t.  Your scope serves as a communication tool about how you will proceed with the project and provides your client an opportunity to react and clarify their expectations about the work.  Similarly your contract lays out a legally enforceable agreement about how you and your client will conduct business together, including key issues such as services offered, payment terms, data ownership, contract termination and renewability.  Should you reach that “worst case” scenario when you and your client reach an impasse, your contract makes clear the parameters to which you’ve agreed.

Rad Resource: For more information about contracts and small business-related legal issues, see Nolo’s Online Legal Forms.

Hot Tip:

  • Hone Those Communication Skills.Sometimes there are client-consultant disagreements about how a project should proceed, even after the contract has been signed.  These moments call for strong communication skills: listen actively to your client, state your positions clearly, manage strong emotions (yours/your client’s) and maintain professionalism.  Remember, conflicts often arise from differing perception of a situation rather than objective facts; it’s important to be able to take the client’s perspective.  Make your goal about coming to a mutual agreement.

Rad Resource: see HelpGuide.org’s conflict resolution skills.

Hot Tip:

  • Be Clear on Your Own Standards. When the client’s expectations about the project change between start and finish of the work, it’s important to be clear about your own standards by writing them down.   Consider the following:
  • Logistics & Scope Changes: How does this impact your project’s time frame, budget and staffing?  Where can you be flexible and where can you not?  Do alterations erase company profits; place too great a burden on your time/staffing capacity?
  • Work Quality/Integrity & Scope Changes: Do requested alterations reduce the quality or rigor of data collection, create conflicts of interest, and lessen the impact of your work?  In some cases these decisions are clearly outlined by professional standards, while other times we must develop our own professional standards.

Rad Resource: See AEA Guiding Principles for Evaluators.

The American Evaluation Association is celebrating the Chicagoland (CEA) Evaluation Association Affiliate Week with our colleagues in the CEA AEA Affiliate. The contributions all this week to aea365 come from our CEA members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.


· · · ·

Greetings I am Mark Griffin. At the time of writing this article I am fortunate enough to be in the middle of a world trip. Key events of my trip: last week I was in Fiji chairing the Pacific Conference for Statistics and Information Systems, my third trip to Fiji, with a rapidly developing workshop program that I have initiated. This week I am in Adelaide, Australia presenting lessons learnt in Fiji at the Australian Statistical Conference. Last night I held the first event for our societies’ section I founded earlier this year, Section for International Engagement. Tomorrow, I fly to North Korea to present Pyongyang University of Science and Technology and Statistics Without Borders co-organised event.

Working with friends and colleagues in developing nations is a true passion of mine. I have also set up an Australian NGO to deliver further training and consulting.

So what advice would I give to like-minded colleagues who have a similar passion?

Tips:

  • Find a mentor (or several). Working in developing countries is incredibly rewarding, but can also be incredibly demanding. Line up people who can support you through the emotional challenges involved, bounce ideas back and forwards, and celebrate with as you enjoy the fruits of your labour.
  • Make strong partnerships. The concept of partnership is a matter of humility, patience, and acceptance. As an outsider you might have superior academic knowledge, and yet your colleagues will best know what’s happening within their country, the needs and constraints, and will generally be the people who have made the largest personal commitment. Strong partnership requires constant communication back and forth about expectations, underlying motivation, and mutual appreciation.
  • Long-term sustainability is difficult. Many a kind-hearted person has gone in for a short duration and set up some potentially beneficial services (such as housing or healthcare facilities), and then quickly left again only for those services to fall into disuse. Any overseas colleague needs to think about the long-term benefits that collaboration will produce (and whether the benefits that you have in mind match the vision of the local people).
  • Communication, communication, communication. As a person who has recently gotten married I am constantly re-discovering the importance of improving all channels of communication. Constant communication is perhaps even more vital with colleagues living and working in completely different contexts. There are too many promising projects that have succeeded or failed, primarily due to the quality of the communication between the stakeholders.
  • Personal motivation is crucial. Make sure that a project is one that you personally are motivated about. At the end of the day, projects have joys and challenges, and to remain committed requires that you have personal motivation for the project to succeed.

Resources:

The American Evaluation Association is celebrating Statistics Without Borders Week. The contributions all week come from SWB members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

· · · ·

Hi, I am Laura Gagliardone. For about seven years I have collaborated with multilateral, bilateral and non-profit organizations geographically targeted to Africa, South Asia and the Middle East and focused in the following areas: research, communication, knowledge management, monitoring and evaluation, program planning, development and coordination.

RELEVANCE: I like thinking of evaluation as a way to improve programs and a chance to adjust activities accordingly to the lessons learned. A quality evaluation depends on the relevance of the quantitative and qualitative data collected and on using communication methods to raise stakeholders’ trust, interest, participation and contribution.

HOT TIP: Evaluation and communication are two disciplines which complement each other. For an evaluator, gathering and analyzing quality data is as important as transmitting the findings. Pay close attention to the communication flow among participants in order to create a positive setting where everybody is contributing to the success of the activity and gaining ownership and accountability.

HOT TIP: Position yourself as a networking hub of the program. Focus your interpersonal and communication skills in a way that installs trust, confidence and builds participant commitment. To minimize evaluation anxiety and to get a good overview of the context, an evaluator needs to be perceived as a community insider who becomes integrated and participates in daily activities.

LESSONS LEARNED: In one of the lowest income areas of Nairobi, Kenya, I evaluated an educational program for children. I used communication strategies to enhance the community’s participation and dialogue. For example, before flying to the country, I read about the cultural, economic and socio-political context and started interacting with people in the field using information and communication technologies. Once in the field, I began to empathize with locals by participating in their daily activities. While undertaking the evaluation I preferred face-to-face interviews especially with children; used open-ended questions and answers; and visited families in their houses, and brought little gifts and greeted them in ‘Kiswahili’. People in developing countries often live and work at a different speed than in Western societies, therefore they appreciate when they are given time. I have learned to appreciate peoples’ strong sense of community and capacity to interact and share. I have also noticed that individualism and independence are rarely exercised. Where possible, evaluators are advised to seek one-on-one interactions that elicit personal perspectives. As participatory techniques that elicit visioning and action planning by local community members, consider mapping exercises, supplying articles or short essays, writing brief notes, creating designs and even sharing recipes as analytical tools that inform the initiative. Enabling a participatory setting facilitates acceptance of the evaluation findings and increases ownership of the activities.

The American Evaluation Association is celebrating Washington Evaluators (WE) Affiliate Week with our colleagues in the WE AEA Affiliate. The contributions all this week to aea365 come from our WE members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · ·

My name is Susan Keskinen. I work for Ramsey County (Minnesota) Community Human Services as a Senior Program Evaluator.  My evaluation projects are related to adult services and to employment related services provided to Minnesota Family Investment Program (MFIP) participants, Minnesota’s version of the Temporary Assistance to Needy Families (TANF) program.  I am also the Communications Chair for the Minnesota Evaluation Association.

I am appreciative of the effort and thought that went into creating AEA’s public statement on Cultural Competence in Evaluation.  Having such a statement affirms its significance and places a value on incorporating cultural competence into evaluation work.  My challenge has been how to apply or integrate culturally competent practices into my evaluation work.  I have searched for and found several helpful documents on the internet that take a more applied than theoretical approach to cultural competence in evaluation.  Here are a few resources that I have found to be especially helpful.

Rad Resources:

The Colorado Trust, a grantmaking foundation, has two excellent documents to assist evaluators in applying culturally competent practices to their work.

Rad Resource:

Understanding styles of communication is an important part of cultural competency.  A chapter from the Peace Corps Cross-Cultural Workbook on styles of communication makes distinctions between indirect and direct communication, describes nonverbal communication, and includes exercises and examples.

Twin Cities Hot Tip:

When you are in Minnesota for the AEA conference, check out the Minnesota History Center located at 345 W. Kellogg Blvd. in St. Paul.  In addition to its great exhibits and library, it has a café (open through lunch) and two museum stores. It is located near the Cathedral of Saint Paul and the Minnesota State Capitol.

The American Evaluation Association is celebrating Minnesota Evaluation Association (MN EA) Affiliate Week with our colleagues in the MNEA AEA Affiliate. The contributions all this week to aea365 come from our MNEA members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

·

Hi, my name is Bikash Kumar Koirala. I work as a Monitoring and Evaluation Officer in the NGO Equal Access Nepal, which is based in Kathmandu, Nepal.  I have been practicing monitoring and evaluation work for over five years, which is focused on development communication programs.  A research project that EAN has collaborated on Assessing Communication for Social Change (AC4SC) developed a participatory M&E toolkit based on our experiences.  One of the modules in this toolkit is the Communication Module, which is summarized as follows.

As a result of AC4SC, the communication systems in our organization improved a lot and became more participatory. We began to understand that effective communication and continuous feedback is essential to the success of participatory M&E. Communication inside organizations and outside can be quite challenging sometimes because different people have different perspectives and experiences.

Lessons Learned

Community Involvement: After the AC4SC project, the level of engagement with communities by the M&E team increased considerably. Their involvement in ongoing participatory research activities and providing critical feedback has proved very useful to our radio program development. This has increased community ownership of our programs. As well as work undertaken by the M&E team, this research is conducted by network of embedded community researchers (CRs).  These activities have produced research data, which is analyzed and triangulated with the other sources of data (such as listeners’ letters) to produce more rigorous results.

Internal Communication: Regular constructive feedback related to program impact and improvement is given to content teams by the M&E team.  This has increased dialogue and cooperation between the M&E and content team members.  Before the AC4SC project, content team members didn’t usually take M&E findings into account because they felt that they already knew the value of the program content through positive feedback from listener letters. The value of M&E has now been recognized by the content teams. They now ask for more in-depth data to generalize feedback they receive. The M&E team addresses this through research and analysis using many different forms of data from varied sources.

Use of New Communication Technology: The M&E team has been analyzing SMS polls, text messages, and letter responses, and triangulating these with the CRs research data and short questionnaire responses to present more rigorous results to program team members, donors and other stakeholders.

Some Challenges: In participatory M&E it is important to understand the roles of everyone involved in the process. Effectively presenting results for better communication and the utilization of M&E findings among different stakeholders is an ongoing challenge. Time to effectively undertake participatory M&E and is also an ongoing challenge.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · · · · · ·

Our names are Eun Kyeng Baek and SeriaShia Chatters and we are an evaluation team and doctoral students at the University of South Florida. The dynamics of a metaevaluation team can determine the overall success of a metaevaluation. Program Evaluation’s Metaevaluation Checklists help guide a metaevaluation, however the dynamics of the team must be considered when leading a metaevaluation. Here we will share a few helpful hints to help improve the dynamics of a metaevaluation team and ensure a smooth, successful metaevaluation.

Lesson learned

Communication: During metaevaluation meetings, observe and listen more than you talk

It is important to understand the ‘how’ and ‘why’ behind your team members’ communication styles. Although you may be familiar with each member’s communication style outside of the team, understand that the dynamics of the team can alter individual’s communication styles and undermine the success of the metaevaluation. Since nonverbal communication dominates 75% of a person’s message, observing your team members during a meeting can provide cues to possible problems within the inner workings of your team. Encourage candid, open communication balanced with professionalism and respect for each team member.

Diversity: Embrace the diverse backgrounds of your team and utilize areas of expertise

Each team member will bring their culture, expertise, and knowledge to the table. Embrace these differences and use them to strengthen the team and the outcome of the metaevaluation. Empathetic listening is an important technique to use. Empathetic listening involves listening to understand your team member’s worldviews and allowing yourself to see the metaevaluation from their point of view.

Leadership: Recognize team member strengths and limitations

Team leaders should recognize team member strengths and limitations, and ensure each team member is assigned a task(s) that utilizes their strengths. Understanding team member roles is beneficial information to be added to the toolbox of any metaevaluation team leader. Some of the advantages of understanding team member roles include increased team effectiveness; increased team cohesion; a better understanding of the underlying dynamics associated with working in a team; and the possibility of profit increases due to better productivity.

Conflict: Employ effective, ethical methods to diffuse conflict

Team leaders should employ effective and ethical methods to diffuse conflict when they recognize difficult team members. Some useful conflict resolution techniques to have in your tool box are the art of persuasion, smoothing, and conciliation. Persuasion includes providing the other side with factual evidence on a position’s correctness and pointing out how the proposition will benefit the other side. Smoothing and conciliation involve emphasizing the similarities of two parties, pointing out common philosophies, and avoiding negative interaction. The key here is to reduce tension and increase trust between two parties.

Here’s to a smooth, successful metaevaluation!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · ·

My name is Oscar Figueroa and I serve as a professor-researcher at Colegio de Postgraduados Montecillo Campus in Mexico.   In 2005 a number staff members of Colegio had the opportunity to attend a workshop with Dr. Fetterman in empowerment evaluation, since then I have been taking advantage of the empowerment tools and methodology to achieve community development.

My research interests include community capacity building and decision making, which involves a lot of effort from community members in rural Mexico, since it has been proven to be one of the most difficult goals in order to make a better use of natural resources and economic integration.

In my experience some of the elements necessary to get the most out of the empowerment workshops with community members are:

Hot Tip: Empowerment evaluation as a communication tool. One of the advantages of empowerment workshops is the “feeling” (perception) community members have that they are actually being taken into account and listened. It is very important as an empowerment facilitator to make sure all participants have clear that, whatever the objective of the workshop is (e.g. deciding about relevant projects, or natural resources management, or organization for production) the objective of the way it is conducted is to LISTEN to them and be clear about what each and all of them perceive as important, nobody is left un listened.

Hot Tip: Empowerment evaluation as a negotiation process. It is very difficult to come out with a common perception of what the main priority would be regarding development issues in any given community. That is why explaining to the workshop assistants the relevance of the empowerment method providing a way of taking into account each and every single participant standpoint is necessary. This way, everybody feels listened and taken into account and at the end of the day, even though their own idea was not the most relevant, people commit as a group on what has been “negotiated” as what is best for them all in the long run and acts accordingly.

I look forward to your thoughts and learning more about your experiences concerning community development and empowerment evaluation.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

I’m Norma Martinez-Rubin, an independent evaluation consultant and Independent Consulting TIG Chairperson 2011. Here I present a few lessons learned in transition from being an external to internal consultant.

Claiming to be multicultural boosts the self-confidence required upon beginning a new evaluation project. Combine that with visualization of near success and that increases my confidence by volumes. Whether I use such “tricks” to approach a short-term or multi-year project, my recurrent challenge is to find the right mix of approachability, credibility, contribution, and professional satisfaction to come away from the project with a sense of integrity and professionalism. Arming oneself with professional standards and competencies is a requisite, but that doesn’t necessarily minimize what I call the “people factor” —the intricacies and idiosyncrasies within human relations that undoubtedly influence one’s performance as an evaluator in organizational cultures with their own vocabularies, operational flows, and personalities.

Picture yourself as the evaluator on a project requiring your expertise, flexibility, patience, and skills to discern pertinent issues from personal preferences of project stakeholders surrounding you. What are the agendas and preferences at play?

 

Lessons Learned

Understanding one’s emotions and their management extends beyond the field of psychology into organizational and professional development. Daniel Goleman (http://danielgoleman.info/topics/emotional-intelligence/) expanded the concept of emotional intelligence from it origins within the field of psychology. Since the mid-90s, his writings have been used for leadership development and business development courses. And for good reason. Personal and social competence combined with intellectual competence makes for a better-rounded individual. Our world could stand to have more people who reflect upon that.

Evaluators have recognized the quality of being reflective among the necessary elements that distinguish quality evaluation. Based on a review of manuscripts submitted for publication in a forthcoming book on qualitative inquiry, Janet Usinger at the University of Nevada, Reno and colleagues identified that among the Three Hallmarks of Quality in qualitative evaluation. The other two were transparency and an educative process (Evaluation 2010, Session #314, “Quality In Evaluation: How Do We know It When We See It in Qualitative Evaluations?”)

Hot Tip: Participation in a local chapter of a management consulting organization (www.imcusa.org) exposes one to varied disciplines outside of the field of evaluation. In a collegial and professional atmosphere, learning from management consultants outside of the evaluation field increases one’s appreciation of those “external factors” typically considered when designing a logic model or presenting alternative explanations for evaluation findings. In other words, one’s worldview is expanded. That’s a good thing that transfers to better communication with prospective clients and/or program stakeholders no matter the culture.

The American Evaluation Association is celebrating Independent Consultants (IC) TIG Week with our colleagues in the IC AEA Topical Interest Group. The contributions all this week to aea365 come from our IC  TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · · ·

Hi, I’m Marty Henry, founder and President of M.A. Henry Consulting, LLC in St. Louis, Missouri.  As evaluators in a small, independent evaluation firm, the evaluation team at M.A. Henry Consulting is often asked to join others in joint evaluation ventures. We look forward to these invitations and are anxious to move forward with potential partners. However, we have learned to step back and take stock before we agree to any collaboration or partnership. We put on our evaluator’s hats. We use the data collection and analysis skills we have to evaluate whether the partnership will be rewarding for both us and the other party before committing.

There are three steps we take before agreeing.

Cool Trick #1: Talk several times with the primary person with whom we will be interacting. We send emails with questions and responses. We determine if communication between the two of us is clear, on-topic, pleasant and timely.

Cool Trick #2: During the conversation, we examine the quality and depth of evaluation knowledge and skills of our potential partner. We discuss the approaches they have taken in the past to similar evaluations and if they are compatible with our philosophy for approaching such evaluations.

Cool Trick #3: We clarify financial and other contractual agreements, specifically in terms of identifying the fiscal agent, publication rights, payment schedule and roles and responsibilities of each partner.

If we are uncomfortable with any of the three steps we do not usually move forward with the collaboration. You may have other areas that are critical to your working relationships with others. If so, include them in this list.

Collaborations can be wonderful – both in terms of working with new, exciting evaluators and in gaining insight into new approaches to evaluation. They can also be unpleasant. Identifying and evaluating the elements that are key to you in successful collaborations will enhance the possibility of the former and reduce the possibility of the latter!

The American Evaluation Association is celebrating Independent Consultants (IC) TIG Week with our colleagues in the IC AEA Topical Interest Group. The contributions all this week to aea365 come from our IC  TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · · ·

My name is Maran Subramain and I am a graduate student at Western Michigan University. I served as a session scribe at Evaluation 2010 and attended session number 577, Communication: At What level Does It Help or Hinder Evaluation Capacity? I chose this session because, as a fairly new graduate student in the evaluation program, I am interested in understanding the competencies that are needed by an evaluator, and communication or interpersonal skill is one of them.

Lessons Learned: This session explained about how volunteer board members can better serve in an evaluation association; and the challenges and benefits of small school district program evaluation. Here are some of the substances that I gained:

  • Small evaluation associations should limit the number of projects they are involved in and should form a strong committee for some of the projects they are involve in. By doing so, the association can better focus and manage the projects.
  • Attending a lot of board meetings and replying to a large volume of emails could be a burden to volunteer board members. Closer attention must be given to communication during the initial stages of board meetings or in emails so that ineffective communication between volunteer board members and information seekers can be avoided.
  • The second presentation explained the challenges and benefits of a program evaluation conducted in a small town in central Florida. The evaluators benefitted from direct communication with major stakeholders, easy access to schools, the program, and families, and less bureaucratic decision making.
  • Challenges encountered included the evaluators being viewed as ‘outsiders’ to the program when they were collecting data, and changes were hard to make due to various parties’ personal interests.
  • Small scale program evaluation can be as challenging as, or more challenging than, in a large school setting.

Hot Tip: I think there is a great potential for this topic to be developed more. For example, will any particular skills such as positivity and openness make the evaluator-stakeholder communication better? Or, can nonverbal cues such as ‘toning down’ the evaluators’ dress code when they visit very poor families help evaluators to gain more information? These ‘soft skills’ are not explored widely in evaluation, and they may bring lot of benefits if it is studied properly.

At AEA’s 2010 Annual Conference, session scribes took notes at over 30 sessions and we’ll be sharing their work throughout the winter on aea365. This week’s scribing posts were done by the students in Western Michigan University’s Interdisciplinary PhD program. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice.

· · ·

<< Latest posts

Older posts >>

Archives

To top