Hello, I’m Sondra LoRe, Ph.D. Since 2014, I have been learning from conducting focus groups, interviews, and observations with educators and students in virtual spaces. The COVID-19 pandemic has undoubtedly amplified the use of virtual spaces for evaluation. Before joining the National Institute for STEM Evaluation & Research (NISER), I worked as an educator evaluator, teacher, and school principal, so I’ve been on both sides of the virtual pond. Here are some tips that work to make the most of virtual spaces for evaluation and teaching.
Hot Tips:
1.) Less is More. Focus groups in a virtual space, even with the ease of video recording, are complex with a group of 6 or more. Grab a colleague or two and leverage the use of breakout rooms. Making groups small increases the participation level and engagement.
2.) Infuse visuals. There is so much truth in the expression, “a picture is worth a thousand words.” Leverage the “share screen” option in virtual meetings to engage people. The visual could be materials used in the program, a map of activities, or a list of intended outcomes for discussion.
3.) Observe overtly. There is no hiding behind a ficus plant in the corner of a classroom in an online space, so make the most of it! Be purposeful! Ask the hosts to introduce you and then share your goals and purpose for the observation. Again, make use of those breakout rooms, observe and record.
4.) Document discourse. Make the most of the chat in virtual spaces. Following a discussion or observation, the saved chat can be a resource of rich information about what people value and need in the program. Leave your contact information in the chat to encourage folks to reach out to you after the meeting.
Lessons Learned:
Your brain will try to make sense of your face on the screen, so if you find your eyes gravitating toward your face as you speak or observe, change the display settings so your face is hidden. That will help you keep focused on others.
Cool Trick:
Smile and wait, wait, wait. Responses to questions in virtual space will take a bit longer for various reasons, particularly finding that unmute button! Slow your roll by extending your time a bit long for focus groups and interviews in a virtual space. If the background is quiet, consider asking participants to leave their microphones unmuted so people can respond “popcorn style.”
Rad Resources:
Learn more about the virtual platforms you are using by checking out books for educators on the topic. The more you understand the virtual platforms, the better you will be at using the spaces effectively.
Teaching Effectively with Zoom: A practical guide to engage your students and help them learn by Dan Leavy https://www.teachingeffectivelywithzoom.com/
Holly Clark has some great books about using Microsoft Teams and Google Classrooms on her Infused Classroom website https://www.hollyclark.org/the-books/.
The American Evaluation Association is celebrating Ed Eval TIG Week with our colleagues in the PreK-12 Educational Evaluation Topical Interest Group. The contributions all this week to aea365 come from our Ed Eval TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
Hi, this is TY. I am currently taking a course on the program evaluation of the program as a part of my Professional Master of Education degree at Queen’s University. Your post on the practical approaches towards program evaluation in a school setting was very interesting to me. Many of the strategies you mentioned are relevant to my current situation as I, as a participant and an evaluator, conduct a lot of meetings and evaluations virtually. For instance, the chat feature provides participants with a risk-free environment, thereby creating opportunities for more authentic responses than oral responses. In addition, I appreciate you mentioning working in small groups and visual aids, as it creates a more engaging and goal-oriented evaluation setting. I am curious to know about the potential benefits of having an overt observation during the evaluation process, as one might think whether the presence of an evaluator and transparency of mentioning of the goal of the evaluation creates anxiety or might even change the participants’ behaviours that could potentially jeopardize the integrity of the evaluation outcome. Thank you for your post!
Hi Jamie! Thank you for reaching out! I am just seeing this comment – July 21th so sorry for the delay in responding. When I am observing, particularly in a virtual space, people do likely change their behavior at first- but then the effect often goes away. After some time I become a part of the landscape. I am always careful to note these things in my evaluation reporting to be transparent, but over time people become accustomed to me watching as we w build our relationships. We collect consent for all of our observations and when working with students under the age of 18, parent consent is needed. My institute currently evaluates about 28 projects all over the US. Much of the work we do is with STEM education programs most of which are with graduate student programs but we do have some K-12 programs. It’s exciting work! Thank you for your questions!
Hi Sondra,
I am currently completing a Professional Master of Education Degree at a University in Canada, and am partaking in a course all about program evaluation. I must say it is something I knew virtually nothing about, and I have learned a great deal about the extent to which evaluation can be used and how it can fit so many diverse programs. I really liked how you wrote an article regarding how program evaluation can be done online, especially for elementary students. The current context I am in is teaching virtually, so many of the strategies you mentioned resonated with me. For example, using the chat function is an amazing way to get students to participate “risk-free” – it works the same for evaluations as it probably means you are getting very authentic answers (more so than if students were answering orally). I also think it is great that you mention using small groups. I can imagine it would be quite challenging to dissect the data collected from a recording of 20 participants as so much is happening at one time. One question I have is about observing overtly. Do you worry that telling the teacher and students exactly why you are there may influence how they act? Do you think students might change their behaviour and say things they would not have normally said because they worry or want to say the “right” thing? The theme of this week in our course is attention to use of evaluation data, and thus the idea of unbiased evaluation data is at the forefront of my mind. As well, have you ever used the “popcorn” style answering option? How did that work? When I have done that with my students, it ends up being too many people speaking at once. How would you analyze that data? Finally, what are the ramifications for collecting data about students? Did you need to receive formal parental consent in order to see the students and record them? I can imagine that this would take quite a bit of work and resources. How big is your team of evaluators?
Hi Marisa and Samantha,
I enjoyed reading your post.
I am currently completing my master’s degree in education, and one of the topics we have covered is curriculum development and the theoretical approaches that support different instructional strategies. Your research and lessons learned to promote a progressist, learner-centred curriculum design.
Although most teachers have attempted to progress from an essentialist approach to instructions to a progressist or re-constructionalist, I still find many teachers unable to move away from their traditional (outdated) instruction methods during this pandemic.
Lessons Learned
1. Partner with students to understand their experience.
2. Meet students where they are.
3. Prioritize teacher-student relationships.
4. Offer “meal-prep” assignments.
The Lessons Learned are great ways to increase student engagement; if children have an interest, then education happens (Mitra, 2010). Point 4 of your lesson learned offer meal-prep, is a great learning strategy that will likely boost intrinsic motivation and a drive to learning.
Thank you for your valuable post
Best,
Marlyn