AEA365 | A Tip-a-Day by and for Evaluators

TAG | video

Hello! My name is Miki Tsukamoto and I am a Monitoring and Evaluation Coordinator at the International Federation of Red Cross and Red Crescent Societies (IFRC).

What if video could be used as the “spark” to increase the engagement and interest of communities in your programmes?

Recently, I had an opportunity to be part of a PVE team for the Global Framework for Climate Service’s programme which aimed to deliver and apply “…salient, credible and actionable climate services towards improved health and food security in Malawi and Tanzania.” To ensure better use and acceptance of this PVE for future programming, IFRC piloted the Most Significant Change technique[1](MSC), using the OECD/DAC criteria of relevance/appropriateness, effectiveness, coverage, sustainability and impact as themes for group discussions. Here are some of the lessons learnt:

Lessons learned:

Rad Resources: PVE videos were made at the community level, the country level and the multi-regional level.

Country level PVEs:

(https://www.youtube.com/watch?v=fSXj0IllfvQ&index=3&list=PLrI6tpZ6pQmQZsKuQl6n4ELEdiVqHGFw2)

(https://www.youtube.com/watch?v=mFWCOyIb9mU&index=4&list=PLrI6tpZ6pQmQZsKuQl6n4ELEdiVqHGFw2)

Multi-country PVE:

(https://www.youtube.com/watch?v=HzbcIZbQYbs&index=2&list=PLrI6tpZ6pQmQZsKuQl6n4ELEdiVqHGFw2)

A Red Cross Red Crescent Guide to Community Engagement and Accountability (CEA)

Guide to the “Most Significant Change” Technique by Rick Davies and Jess Dart

[1] http://www.mande.co.uk/docs/MSCGuide.pdf

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

I’m Emma Perk, Project Manager at The Evaluation Center, Western Michigan University. I find evaluation to be very informative and important, but feel that there is a disconnect when it comes to sharing and implementing the results. I am very interested in data visualization, but today I am here to talk about another avenue—video.

AEA6

Video is a great method for distributing evaluation knowledge and results. This is a not a revolutionary idea (I know) but I have located some great resources for making short videos that are high-quality and low cost. It requires you as the evaluator to boil ideas down into a simple summary, then get a little creative.

Rad Resource: Microsoft Movie Maker (PC) or iMovie (Mac) and a cell phone. You may be thinking I am pretty lame right now, but there are some fantastic features on these programs that are already loaded directly onto your computer. Record interviews during your data collection with your phone and embed the interviews with some basic text and graphics to create a high-quality, low-cost video that you can include with your evaluation report.

Lesson Learned: I learned the hard way, so I will share with you now. Embedding a video directly into your PowerPoint presentation allows for an easy transition from slides to video. It’s easy to embed, but you need to have your movie in the right format. Check out this helpful webpage to find out more.

Rad Resource: Adobe Voice is a free app available on the iPad only. This app is so useful and easy to use; you may consider buying an iPad if you don’t already own one. Adobe Voice has a library featuring 25,000 graphics, backgrounds, and music. Then you record your own voice to finish the movie. It’s fast and simple. Find out more here.

Rad Resource: Join me for my session at this year’s AEA conference.

This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org. Want to learn more from Emma? She’ll be presenting as part of the Evaluation 2015 Conference Program, November 9-14 in Chicago, Illinois.

We are Julie Poncelet, Catherine Borgman-Arboleda, and Jorge Arboleda of Action Evaluation Collaborative, independent consultants who use evaluation to strengthen social change. We want to share our experiences using participatory video (PV) in evaluations with youth.

PV is a dynamic, powerful approach whereby youth use video to capture everything from their stories of change to issues that affect their everyday lives to ideas they have to effectuate change in their communities. As pictured below, we recently engaged PV with a group of teens from a community-based NGO in Yucatan, Mexico. Youth produced videos about their dreams and senses of identity. PV is a compelling approach to explore these themes, which emerged from a Theory of Action process with the NGO; specifically identified was the need to have youth analyze critically their communities and find their voices.

PV positions youth as researchers and evaluators of their own communities and supports them to contribute creatively and critically to issues. With PV, youth design, direct, film, and edit videos. They experience empowerment, ownership, and self-esteem rarely garnered from other evaluation approaches. Adults provide technical assistance, build capacity, and facilitate a process for PV to unfold (not to take over the process!). For evaluations, PV creates a space for community members and stakeholders to see the interests and needs of youth in the community and provides a unique platform to reflect collaboratively on meaning and implications.

p2 p3

Lessons Learned: Focus should be on learning the technology and the video storytelling process, as well as providing an appropriate approach for young people to collectively reflect on themselves and their realities. Give youth time to feel comfortable with the equipment and with engaging others in conversation. And remain aware of group dynamics! We often find that boys are more comfortable and will take leadership with technology, so consider breaking groups up by gender.

Consider using a short set of questions that can be asked by youth to stakeholders included in their videos. The insight can help to contextualize the analysis and overall sense-making. The PV process is as much an outcome as the product; engaging in PV is transformative, so don’t worry about getting ‘perfect’ videos.

Hot Tips: Although building a participatory video kit is not cheap, all you really need is a small camera (preferably with projecting capabilities) and a good quality hand held microphone. We have found that when young people hold the microphone, they feel more empowered to speak, so it helps them to find their voice.

Rad Resources: The PV approach aligns nicely with other qualitative methods (Most Significant Change) and different types of evaluations (Monitoring & Evaluation).

The American Evaluation Association is celebrating Youth Focused Evaluation (YFE) TIG Week with our colleagues in the YFE AEA Topical Interest Group. The contributions all this week to aea365 come from our YFE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

 

·

I am Megan Greeson, an Assistant Professor of Clinical/Community Psychology at DePaul University. I conduct collaborative, utilization-focused program evaluations with community-based organizations and I also teach a graduate seminar course in Program Evaluation.

I am Adrienne Adams, an Assistant Professor of Ecological-Community Psychology and Program Evaluation at Michigan State University. I use collaborative evaluation approaches to help human service organizations build evaluation capacity and foster organizational learning.

We are interested in helping community organizations engage with program evaluation and evaluation findings in order to promote ownership over the evaluation process and findings, and to increase evaluation capacity and use. The ways in which we communicate with communities about evaluation and evaluation data matter and can improve their retention and understanding of important material.

Hot Tip: Multimedia reports (involving text, audio, and video components) can be a great way to reach evaluation stakeholders from a distance. The addition of audio and video to a text-based report can draw attention, create a connection to the evaluator, and help stakeholders interpret difficult technical topics.

Hot Tip: Since multimedia reports can be viewed or heard on a variety of devices at times and locations most convenient for evaluation stakeholders, they can be a good way to reach busy stakeholders with little time to digest a written report.

Hot Tip: Audio and video components can be easily edited to tailor parts of a presentation to individual sites or stakeholders, or to correct presentational errors, like “ums” or animation glitches.

Hot Tip: Adding a worksheet or activity that the audience completes while watching the presentation can assess retention and/or encourage a plan for use.

Rad Resources: Jing, Camtasia, and Adobe Presenter are tools that can be used to create presentations that involve text/figures, audio, and video components.

Jing: http://www.techsmith.com/jing.html  (FREE!)

Camtasia: http://www.techsmith.com/camtasia.html (Free trial)

Adobe Presenter: http://www.adobe.com/products/presenter.html

Rad Resource:  Attend our demonstration “Technological Tools for Creating Multimedia Evaluation Reports” at AEA 2013 to see how we have used these tools in our own work.

This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from theAmerican Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org. Want to learn more from Megan and Adrienne? They’ll be presenting as part of the Evaluation 2013 Conference Program, October 14-19 in Washington D.C.

·

Hello! We are Hanife Cakici, Christopher Pleasants, and Shipi Kankane from the Minnesota Evaluation Studies Institute (MESI) at the University of Minnesota. We were a part of the team of conference evaluators for Evaluation 2012 in Minneapolis. We asked conference participants to select a question from a menu, and videoed their responses to be shared with the rest of the evaluators during the conference. We were able to capture thirty responses and are excited to share our thoughts with you.

Lessons Learned:

  • Love it or leave it: Most of the evaluators we approached agreed to be videoed. There were many others, of course, who were not comfortable being on camera. To comfort our participants, we put our “Minnesota Nice” hats, and acted gregariously by allowing them to use the front-facing camera as a mirror to fix their hair or tie.
  • Privacy rules: We asked evaluators’ permission to share their videos on the AEA’s YouTube channel. Many did not concede. We assured them that the videos will be compiled in password protected, private channel that only the team has access to.
  • Influx of Positive Responses: Videos are more likely to entice positive responses than other means of data collection. Evaluators often did not appear to publicly (or at all) share negative responses on camera. This ought to be taken into consideration when using videos as an interactive evaluation tool.

Hot Tips:

  • God bless technology: iPads have been extremely helpful to take quality videos, and instantly upload them to YouTube. They are easy to use, easy to carry around, and super fun to play with! We are grateful for College of Education and Human Development at the University of Minnesota that allows students to check out iPads for free for up to two weeks.
  • Evaluators in groups: We found out that we are less likely to be rejected when we approach evaluators in a group, which created pressure to respond to our “simple” questions. When evaluators were alone and especially eating something, we were most likely to be rejected. Make sure to understand evaluators’ ecology before you approach them.
  • Adjust your YouTube settings: You can upload videos to YouTube with any of the followings settings:
  • Public: Searchable to anyone on YouTube
  • Unlisted: People with URL can view video.
  • Private: Sign-in required viewing videos.
  • Make yourself visible: Conference participants needed to know ahead of time who we were and what we wanted to accomplish with our short video-questionnaire so that they would entrust their responses and images to us.
  • Have fun: Check out our Evaluation 2012 Video! Thank you all for your contributions!


We’re learning all this week from the University of Minnesota Innovative Evaluation Team from Evaluation 2012. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

·

Oct/11

22

Susan Kistler on the AEA Public eLibrary

My name is Susan Kistler and I am the Executive Director for the American Evaluation Association. You likely know that AEA’s annual conference is coming up the week beginning October 30. What you may not know is that conference presenters are strongly encouraged to share handouts and supplementary materials via the AEA public eLibrary.

Rad Resource: Watch the AEA Public eLibrary over the next couple of weeks as the materials from the conference are added. There are already over 800 slide sets, papers, instruments, and such in the eLibrary and this number will increase significantly in the coming weeks.

Hot Tip: Items in the AEA public eLibrary may be searched and downloaded by anyone – the eLibrary is provided as a public service to the field. Use the “Search Library” link on the left column in the eLibrary to search by keyword or by items tagged for any of AEA’s 40+ Topical Interest Groups, to find items that match your needs and interests.

Hot Tip: You don’t need to be a presenter to share items in the eLibrary. If you are an AEA member, consider sharing instruments, syllabi, reports, etc.

Hot Tip: If you are a presenter, upload your materials prior to the conference and then encourage attendees to access your handouts or slide deck in the eLIbrary rather than printing and distributing handouts on site. At the conference, distribute only those materials that will actually be used during the session, presentation copies and supplementary materials can have a great home in the eLIbrary.

Hot Tip: If you have not uploaded to the eLibrary before, the following short (4 minute) video will walk you through the process.

This contribution is from the aea365 Daily Tips blog, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org.

· ·

My name is Corey Newhouse and I’m the Founder and Principal of Public Profit. We help youth service and educational organizations measure and manage what matters. I’m a self-taught user of video in evaluation, and am writing to share some of my lessons learned.

We have incorporated video into our evaluation studies for the last two years, and have loved the ways in which it has enriched our process and our findings. I have gotten much better at using video purposefully in evaluation, moving from a “shoot-it-and-see!” approach to a more intentional “let’s-film-this-because…” approach.

Hot Tip: Consider your purpose when incorporating video. This makes your shoot days more effective and makes it easier to sort through the footage later on.

1. Do you want to illustrate a process or method? Many complex, expert-driven processes, like teaching and coaching, are very hard to describe accurately. Video can provide a rich record of how the process or method functions from start to finish.

2. Do you want to promote reflection and analysis by a video’s subject? Video has been used for years in training programs as a means to promote self-guided practice improvement and targeted coaching.

3. Do you need a very accurate record of a conversation or interaction? We videotape important conversations and meetings among subjects so that we can generate transcripts that include verbal and non-verbal data. This is especially helpful in conference-style sessions when multiple conversations take place simultaneously.

Rad Resource: An invaluable resource for our work is Video in Qualitative Research: Analysing Social Interaction in Everyday Life from Sage Publications.  This book walks through the preparation, collection, analysis and presentation stages, drawing from the authors’ extensive professional experience.

Rad Resource: For a more theoretically focused take on the role of video in evaluation, check out Barbara Rosenstein’s article, “Video Use for Program Evaluation, A Conceptual Perspective” in Studies in Educational Evaluation 26 (2000). She explores the ways in which video can promote in-depth reflection in the context of program evaluation.

Upcoming Coffee Break Webinars: Please join me on November 10th for How to Get the Clips You Want and again on November 17th for Incorporating Video Into Evaluation Reports. Follow the link to learn more and register.

This contribution is from the aea365 Tip-a-Day Alerts, by and for evaluators, from the American Evaluation Association. Please consider contributing – send a note of interest to aea365@eval.org. Want to learn more from Corey? She’ll be presenting as part of the Evaluation 2011 Conference Program, November 2-5 in Anaheim, California.

·

My name is Yusuf Nazim and I am learning about evaluation from aea365 and from people around the world. I wanted to share today a resource that I have found very valuable.

Rad Resource – The Evaluation Center Videos on Vimeo: The site says “Lively discussions, engaging debates, and dynamic presentations about evaluation over the lunch hour. The Evaluation Cafe is a long-running weekly presentation series held at The Evaluation Center… Presenters generously allow their session to be published and accessible to a wide audience.” They have over 80 videos that you can play, all about evaluation and all are free. Many are an hour long or longer so you can truly learn something.

They focus on many topics. Michael Scriven is in them. The most recent is “Strong Evaluation Plans = Stronger Proposals.”

Thank you to The Evaluation Center.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like submit a Tip? Send it to aea365@eval.org aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

· ·

Archives

To top