Hi, I’m Susan Eriksson, a geologist and science educator reformed as an evaluator for science-related programs. I’ve worked in industry, academia and the non-profit sector and have worked with many evaluators and served on many grant review panels. My tips for today might help you decide if you need to partner with an evaluator who comes from a STEM field.
Lessons learned: Scientists have the content knowledge to delve into important questions needed for significant evaluation. They have insights about the scientific community expectations.
Scientists have the language and culture to work with STEM people. A great deal of ‘eye-rolling’ goes on when a scientist hears the word ‘evaluator’. Recently a scientist kept saying ‘but that’s not proof….”. I calmly replied, “we are looking for measurable results that support multiple and independent lines of evidence that point to significant impact’. He understood the ‘multiple and independent lines of evidence’ because that is the way science works.
Scientists working as evaluators have the ‘trust’ because we have been ‘one of them’. Scientists inherently don’t trust social scientists! I could add an emoticon with a smiley face here, but it is generally true! I know, I have been one of those ‘eye-rolling’ natural scientists.
The other side of the coin: Without additional training, scientists don’t have the skills to do evaluation. As a proposal reviewer, I see random people put on grants as “external evaluators”. Recently a proposal had a mathematics professor as the external reviewer. No credentials, no experience in evaluation….no good rating from me! Good evaluation has standards, ways of thinking, and research methodologies that are normally not part of a person’s scientific training.
Scientists can, however, learn the profession of evaluation, gaining the right knowledge, expertise, and experience. Certificate programs, short courses, webinars, and working with experienced evaluators give many scientists the evaluation skills to combine with content knowledge, trust, insights to potentially do a very effective job in evaluation.
Hot Tip: For evaluators working with scientists, go beyond measurement of generic outcomes to find out what your client’s expectations are. Craft your language about evidence and methodology in terms that are valid to STEM audience. Partner or talk to a scientist/evaluator to gain insights into the rich world of STEM.
Susan Eriksson is a leader in the newly formed STEM Education and Training TIG. Check out our TIG Website for more resources and information.
The American Evaluation Association is celebrating STEM Education and Training TIG Week with our colleagues in the STEM Education and Training Topical Interest Group. The contributions all this week to aea365 come from our STEM TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Pingback: On Scientists Becoming Evaluators | Eriksson Associates
Pingback: On the Education of Stakeholders | Eriksson Associates
Thank you for your post, Susan….
I too have been one of those “eye-rolling” natural scientists, having spent an entire career working as a “hard scientist” in chemistry, I concur with your statement that an easy rapport is formed when scientists speak to scientists; and that rapport works to establish face validity based in a common understanding of language and processes.
However, I also offer that hard scientists are likely to be trained to remove the “valuing” from the presentation of results. I know I was. It is my experience that scientific researchers allow “any user of the information” to use the information in “any appropriate manner.” Let the user decide. Value and context obfuscated…perhaps.
I believe you are describing the fundamental question: what is the difference between research and evaluation? It is my experience that your post is right on the money with respect to the necessity of creating the same common language and processes associated with valuing, as we do in research.
Researchers are not necessarily evaluators…but they could be (given training)!!
I ponder: is it the same experience, the same phenomenon when accountants, lawyers, and individuals in the medical community are added to grant and foundation projects as “external evaluators?” Is it their educational status in a community, that gives them a free pass? Do you see this phenomenon as well?
Again thank you for your post, with my…
Best to you
Michael Kiella