“I can remember stories …,” Anita Baker, my favorite singer-songwriter, reflects on her experiences in the soulful ballad “Fairy Tales”. The lyrics strike a reflective chord. I hear how she was snapped out of happy fairy tales into the snares of reality. But before I tell you about the realities of being an external program evaluator, let me reveal more about who I am.
My name is Angelicque Tucker Blackmon, Ph.D. I work with academic leaders to design and implement human-centered STEM research and evaluation studies and programs that encourage growth, foster success, and promote inclusivity. My 25 years of experience in STEM education evaluation deliver a unique ability to identify and capitalize on the latest innovative learning and assessment practices and thread learning theories into STEM education programs. At Innovative Learning Center, LLC, I leverage my data analytics and ethnography expertise and provide clients with the tools to design, implement, and evaluate STEM education programs that leave a real and lasting impact.
I began with a mention of lyrics to underscore a condition of my work as an evaluator: the snap-to-reality phenomenon. The award letter comes, and everyone has visions of program success, implementation, progress, advancement of knowledge, social innovation, and scholarly productivity. Our minds are filled with visions of happiness, niceties, camaraderie, and collaboration. I have continued employment.
Then, reality sets in and it happens. Side-eye glances, skepticism, sprints to data–or worse, the outright withholding of data–occur. I am not a huge Taylor Swift fan, but I am reminded of the lyrics in her song “Shake It Off”. Things happen when there is a lack of clarity about roles and perspectives. Roles can blur when perspectives are unknown. Murkiness inevitably occurs between evaluators and researchers–until or unless.
Hot Tip:
Unless there are discussions at the beginning, midway through, and near the end about perspectives- and the different ones that evaluators hold compared to researchers- data collection and analysis activities remain foggy, especially to other team members. Often, as the evaluator, I initiate necessary conversations to offer role clarity. I prefer collaborating happily.
The prospective positions dictate the functionality of researchers and evaluators’ roles. Different functions result from maintaining narrow perspectives versus broad perspectives. Researchers take a narrow perspective on one aspect of a program based upon an interest in a slice of the program story. As the evaluator, I take a broad view of the program and narrate all parts of the story, even the parts I am not particularly interested in.
The researcher’s goal perspective is the production of knowledge, whereas the evaluator’s goal is to provide an “independent” assessment of the project and outcomes. Both use standard social science research to generate knowledge but for completely different purposes.
Rad Resource:
“Hello, It’s me.” Adele is a favored soloist, too. Now that you have a sense of who I am and what I do, let’s compose a harmonic refrain to explain the similarities and dissimilarities between research and evaluation. Click here to learn more about us.
Lesson Learned: Discuss evaluator and research roles in the early in a project.
It is critical to discuss the roles of evaluators and researchers within one project during the program development phase. Researchers and evaluators’ perspectives are different, albeit too often unstated. And, even when discussions are brokered during the program design phase, things can go off the rails, especially when access to data is directly tied to advancement. Early, often, and open communication helps avoid confusion and highlights where opportunities exist for both professional roles.
The American Evaluation Association is hosting STEM Education and Training TIG Week with our colleagues in the STEM Education and Training Topical Interest Group. The contributions all this week to AEA365 come from our STEM TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.
Hello Angelicque,
Thank you for your post and for clarifying the differentiation between research versus evaluation. I am currently a Master’s student taking a course on program evaluation, and the difference between research and evaluation is a topic that has drawn my attention. I teach STEM topics in a postsecondary institution, and there is an institutional drive for research-based activities and well-defined and reliable outcomes for our programs. Finding the balance between which activities relate to which role is important for clarifying my position as to when I am producing knowledge versus assessing project impact. In our course readings, Alkin & Taut described how both research and evaluation deal with systematic knowledge, but the context of the knowledge and its subsequent use is what best defines the results of the evaluation (2003, p. 3), underlying your point about the purpose of the knowledge generated being different depending on the goal. Having both roles within a project team seems beneficial, but with clear communication and documentation, could an individual complete both roles within an evaluation?
I especially appreciated how you wove artistic expression and storytelling throughout your article by incorporating lyrics and the feelings produced through songs. I recently attended a conference for college mathematics professors. Our keynote speaker was Sunil Singh, who posited that storytelling and narrative are essential for teaching and learning mathematics. There can be a feeling that STEM topics are boring, but integrating art and more profound understanding is an excellent way of broadening awareness and appreciation. Thank you for doing that for me with your article.