AEA365 | A Tip-a-Day by and for Evaluators

TAG | facilitation

My name is Elizabeth (AKA: Bessa) Whitmore. Now a retired Professor from the School of Social Work, Carleton University, Ottawa, Canada, I have been a member of the Feminist TIG since its inception. The following entry draws on a chapter I am writing entitled “Researcher/evaluator roles and social justice” in a forthcoming Handbook on Feminist Evaluation (edited by Denise Seigart, Sharon Brisolara and Sumitra SenGupta).

Hot Tips:

  • There are a range of roles played by a feminist evaluator, including facilitator, educator, collaborator, technical expert/methodologist, and activist/advocate. Not everyone can do everything equally well, so self-knowledge and confidence in one’s strengths (and limitations) is essential. The personal characteristics, experience and preferences of the evaluator will dictate which role(s) she/he best plays. It is critical to recognize that what role the evaluator plays and how, is intimately tied to her/his own worldview, history, and biography. There is no objectivity; we need to be aware that we are deeply grounded in our own location and life experience.
  • Good “people skills” are essential when engaging stakeholders in the process. These include active listening, cultural sensitivity, non-verbal communication, motivating participants, coordinating relationships, encouraging interactions, supporting others’ ideas, and an ability to reflect critically on one’s own reactions and behavior.
  • Having fun: We should not dismiss the importance of fun in this work. “If I can’t dance, I don’t want to be part of your revolution” said Emma Goldman back in the 1930s. Long hours without some laughter tend to burn people out, or they just drop out.

Cool Tricks: Here are some questions one might ask when planning and implementing a feminist evaluation:

  • In what ways are women (men, bisexual and transgendered people, etc.) treated differently within the program, and how do their experiences and outcomes differ? In what ways do class, race, and gender combine to expand or contract possibilities for participants?
  • Are both women and men being consulted about objectives and activities? Which women, and which men? Has the potential for community resistance to women’s empowerment activities or organizational resistance to female managers been assessed?
  • Did the project have any unexpected (positive or negative) social and gender equity outcomes?

Lessons Learned:

  • A feminist lens enhances validity in all evaluation approaches. For example, an experimental design pays attention to the sample distribution among men and women, considers gender related factors in the questions asked, and in data analysis. A utilization focused evaluation attends to the gender (and other) distribution (in decision-making). Social justice approaches (such as empowerment, participatory, collaborative, transformative, etc.); consider the equality and quality of gender participation.
  • Get involved: A good place to discuss these and other issues is the Feminist TIG.

The American Evaluation Association is celebrating the Mixed Methods Evaluation and Feminist Issues TIGs (FIE/MME) Week. The contributions all week come from FIE/MME members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Hello, we are Carolyn Cohen (Manager, Cohen Research & Evaluation, LLC) and Davis Patterson (Research Scientist, University of Washington Department of Family Medicine). We have partnered on several evaluations over the years, and are always excited to try out new facilitation strategies. We sometimes find ourselves challenged by access to, and time constraints of, potential interviewees or focus group members. As a case in point, we were recently charged with collecting reflections and eliciting new information from a group of participants in a two-week teacher professional development session. We were scheduled for 90 minutes on the agenda at the end of a long day.

Hot Tip: The Interview Design Process, a hybrid of a focus group and a speed dating session, fit the bill. It generated full and lively engagement, allowed for physical movement, and produced findings beyond our expectations. This technique allows the evaluator to collect responses to multiple questions from a large group of people in a short amount of time. Here is how it works.

  • The evaluator generates a question set with 3-5 questions.
  • Participants are divided into groups, preferably seated at tables; each group receives the same question set.
  • Tablemates are assigned a set of interview partners; please see the attachments for the explanation of how to do this.
  • Each participant conducts interviews on one assigned question, and is interviewed on the other questions by tablemates.
  • Interviewing is complete when each participant has been interviewed on each question.
  • At this point, participants move to their “like” tables, and synthesize their findings. (i.e., those who conducted interviews on question 1 gather with other 1’s, etc.)
  • Finally, each group presents their findings to the whole group, and the evaluator can then facilitate a learning discussion.

Rad Resources: In AEA Library

Lessons Learned

  1. This process is for a group of 12 or more participants, but the beauty of it is that there is almost no limit to the number of persons, because they mostly manage their own participation.
  2. Use a visual aid to explain the process to participants. We attached 2 examples.
  3. The process definitely takes careful advance planning. The number of participants determines the number of questions and seating configuration. Depending on the physical space, interview partners can sit across from each other or next to each other (see our two PowerPoint visual aid examples).
  4. Ask participants to write their notes as clearly as they can, and be sure to collect all of them. Electronic note-taking is a great option.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· · ·

My name is Kylie Hutchinson. I am an independent evaluation consultant and trainer with Community Solutions Planning & Evaluation. I am also a regular facilitator of the Canadian Evaluation Society’s Essential Skills Series course.

When I first started my evaluation practice, I was concerned that my statistical skills would be weak. Instead, I quickly learned that group facilitation was where I actually needed to focus the majority of my professional development efforts.

Sam Kaner defines a facilitator as, “an individual who enables groups and organizations to work more effectively; to collaborate and achieve synergy. They are a content neutral party who…can advocate for fair, open, and inclusive procedures to accomplish the group’s work. A facilitator can also be a learning or a dialogue guide to assist a group in thinking deeply about its assumptions, beliefs, and values, and about its systemic processes and context.”

I find that my facilitation skills really come into play when developing logic models and evaluation frameworks with clients. For those practicing Participatory, Empowerment, Developmental, or many other types of evaluation, where client participation and input is important, or where utilization of findings is a concern (and when is it not?), effective facilitation is a critical factor in determining the overall success of the evaluation.

Rad resource:  Facilitator’s Guide to Participatory Decision-Making available from Jossey-Bass. This book is a great resource for those wishing to quickly develop their skills in facilitation, or for those looking for a refresher. The layout is very user-friendly for busy evaluators who have limited time for reading. I regularly pull it from my shelf when I’m feeling rusty and need a quick refresher, or when I’m looking for new facilitation techniques. For something more intense, consider formal training in group facilitation. Better yet, sit in on a session with someone with recognized skills in facilitation. Some of my best learning has come from watching good facilitators in action.

Reference:  Kaner, S., Lind, L., Toldi, C., Fisk, S., & Berger, D. (2007). Facilitator’s Guide to Participatory Decision-Making (2nd ed.). Jossey-Bass.

This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

· · · · ·

Hi all. We are Cindy Tananis and Cara Ciminillo, Director and Project Manager, respectively, of the Collaborative for Evaluation and Assessment Capacity (CEAC) in the School of Education at the University of Pittsburgh (www.ceac.pitt.edu).

Over the years, we have had the opportunity to work with various clients on variety of projects. No matter what the issue or discipline, we have found a technique called ‘The Round Robin’, to be an effective way to collect data in a short period of time from a large group of people. The Round Robin provides an opportunity for participants to offer, collect, analyze, and interpret perception data within a relatively short period of time, as a facilitated activity.

This process works well with large or medium-size groups, young and adult people, and typically focuses on 4 or 5 questions.

The Round Robin set up places people in groups of 8 or 10 with people facing each other in chairs:

Row       Row

A            B

Q1   «  Q1

Q2   «  Q2

Q3   «  Q3

Q4   «  Q4

Q5   «  Q5

Each person in Row A has a unique question sheet (Q1, Q2, Q3…). Each person in Row B also has unique questions that match those in Row A. Each sheet has room for comments and notes.

Data Collection

Each person asks the person across from them the question on their sheet and records notes of those responses. Limit the time for responses…give each person 2 minutes to ask and record their partner’s response. In total, each dyad should need about 5 minutes.

Once the entire group has asked each other their question, row B stands up and moves down one position. Again, each person asks the person across from them the question on their sheet. And again they record the response. This rotation continues until every person has answered every question and also recorded notes from 4 or 5 (depending on group size) respondents.

Data Analysis and Reporting:

After all data are collected, regroup people by joining together all people who collected notes for the same question. As they gather, they ‘debrief’ their experience by sharing common themes and interesting responses. A recorder/reporter shares out the “summary” of their question with the whole group for larger discussion.

Hot Tip: This is a great opportunity to encourage individual input from all participants in a manageable fashion. It also provides for data collection, thematic analysis and reporting back to the larger community in one activity.

Hot Tip: Limiting the time for each dyad question-response encourages people to respond with prioritized information and keeps the activity moving, fresh and fun.

Hot Tip: Gathering the written notes (original and summary) allows for an additional check and documentation for reporting purposes.

Want to learn more from the CEAC staff? They’ll be presenting as part of the Evaluation 2010 Conference Program in San Antonio on Wednesday, November 10, from 4:30-6pm in Republic C.

·

My name is John Nash, and I am an associate professor at Iowa State University in the Department of Educational Leadership and Policy Studies with a joint appointment in Human Computer Interaction. I’m also a program strategist, evaluator, and design geek.

Today I’d like to share ways to improve slide presentations.

Hot Tip: Know Your Audience – This is an oft overlooked tip from Nancy Duarte, author of slide:ology, a wonderful book on the art and science of creating great presentations. Duarte suggests seven questions to ask before developing any presentation:

  1. What are they like?
  2. Why are they here?
  3. What keeps them up at night?
  4. How can you solve their problem?
  5. What do you want them to do?
  6. How can you best reach them?
  7. How might they resist?

It’s easy to see how these questions would be important to answer in a business or sales presentation. However, amongst evaluators, they are often overlooked when designing a client briefing or conference presentation. I’m especially drawn to question 5, which reminds me that every presentation should be a call to action.

Hot Tip: Let Go of Text – Text can be a crutch for the time-pressed and insecure presenter. Duarte suggests three strategies to excising text as a crutch on your slides:

REDUCE: Practice presenting your slides a few times, then highlight one keyword per bullet point. Deliver your slides from only the keywords, using the rest as notes. Eventually, consider replacing the keyword with an image.

RECORD: Read your presentation out loud and record the audio. Play it back. Once you get over the horror of hearing your own voice, you’ll be able to concentrate on your content and not focus on the slides.

REPEAT: Practice, make note cards, draw a mind map, do anything that helps you visualize or create a cheat sheet. Then, look at your slides and delete as much as possible that’s covered already on your cheat sheet.

Rad Resources: If I could recommend only two books on presenting, they would be the aforementioned slide:ology and Gary Reynold’s Presentation Zen.

Hot Tip: Ignite! Ignite-style presentations are exactly five minutes long using 20 slides that auto-advance every 15 seconds. Using Ignite means delivering the most salient content, from a point of passion, while remaining story-focused (and thus, I argue, more audience focused). For example, watch Molly Wright Steenson’s presentation on the otherwise arcane topic of pneumatic tube networks. Did you adsorb more information than in any other five minutes of your day? Notice how she uses minimal text, good images, and a great story to grab your attention.

Want to learn more from John about giving great presentations? He’ll be offering an AEA Coffee Break Webinar on Moving Beyond Bullets: Making Presentation Slides Compelling on April 15 as part of AEA’s Coffee Break Demonstration Webinar Series (free for AEA members!). Learn more at http://comm.eval.org/EVAL/coffee_break_webinars/Home/Default.aspx

· · ·

My name is Cassandra O’Neill and I’ve been a consultant for the past 10 years. I’m a member of a network of consultants and coaches called Wholonomy Consulting.  I’m also the President-Elect for the Arizona Evaluation Network and a member of the AEA Local Affiliate Council. A theme in my work is using effective engagement for high impact collaboration.  I have several resources to share with others interested in increasing the effectiveness and impact of collaborations.

Hot Tip: Collaboration is one of those phrases like strategic planning – that can mean something different to every person. Collaboration is talked about all the time without any clarification or definition. One of the most useful things to do with a group that is collaborating or wants to – is to explore the mental models that people have about collaboration and related ideas such as resources.

I have a pretty quick and easy way to do this which helps groups identify the different beliefs and assumptions about collaboration that are in the room. Why is this so important?  To answer this question, I am going to tell you a story about something that happened recently. I just gave a workshop on high impact presenting. In this workshop I demonstrated about 12 different ways to present information without lecturing or telling people the information.  At the end of the workshop I was reading the Appreciative Reviews and someone wrote – I didn’t learn anything that I can use in presenting.  I realized that this person’s mental model of presenting as lecturing was so strong, that nothing that was experienced in the two hours could be connected to presenting — since it wasn’t lecture. Think about the missed learning opportunities!

An example of how important exploring mental models, beliefs and assumptions can be for effective collaborations — is the impact of the beliefs that people have about resources.  When I ask people to identify a belief they have about resources, there is usually one of two responses shared.  Either the belief that resources are unlimited and abundant is shared or the belief that resources are scarce and limited. You can see how knowing people’s beliefs about this topic influence everything else that is involved in collaboration.  This exploration of beliefs may allow people who believe resources are scarce to see that this may be a limiting belief rather than a fact. See a link to my blog for tips on how to do this with a group.

http://bit.ly/mentalmodels

And you can use any word or set of words with a group. Once I had people do this exercise three times with the following words – collaboration, resources, and sustainability.  You could do it with the word evaluation and quickly learn about the beliefs and assumptions that people have about evaluation.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

· · ·

My name is Jara Dean-Coffey, and I am the principal and founder of jdcPartnerships (www.jdcpartnerships.com). We partner with our clients to build their adaptive, strategic and leadership capacity and do so by using an evaluative inquiry approach to strategy formation, evaluation and assessment, leadership development and technical assistance and training. Our clients are in all sectors with the unifying purpose of striving to affect positive social change. Because we work in the intersection of disciplines, a lot of our initial work is about helping our clients get clear.

Hot Tip: Developing and using group process and facilitation skills is a important part of our tool kit and we are constantly finding ways to enhance. Often when working with a full organization, across organizations or community wide, we use a graphic recorder. A graphic recorder is, in short, a visual practitioner. The use of color, spatial placement of words, images and concepts has been incredibly powerful in our work around strategy and evaluation. The bonus: Clients LOVE IT. Instead of getting a 20 page bulleted report of a retreat, we give them a graphic record reflecting their words and process and a cover memo summarizing themes, recommendations and next steps. And for those who could not be at the meeting, they get a sense of the flavor of the conversation and its spirit and soul. Because ultimately, we are people doing this work. If you’d like to see an example created for our firm during a 2-day retreat to develop a Theory of Change with the full staff of a mid-size social sector organization, I have uploaded one into the AEA eLibrary at http://bit.ly/jdcGraphicFacilitationExample.

Rad Resource: Check out the International Forum of Visual Practitioners Website (http://www.visualpractitioner.org/) for more information on the history and range of skill that they can bring to a process. Another key resource is Grove Consultants (www.grove.com). They not only provide graphic facilitation services but a great range of tools that you can integrate in to your practice. They also have an affiliate program where you can find a consultants to support your work.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the
American Evaluation Association
. Please consider contributing – send a note of interest to aea365@eval.org.

· ·

Archives

To top