I am Dr. Kelly Skinner, an independent evaluation consultant and Assistant Professor at the School of Public Health and Health Systems at the University of Waterloo (UW) and program lead for the Master of Health Evaluation, and Jasmin Bhawra, a PhD Candidate at UW and evaluation consultant with the Saskatchewan Ministry of Health. Our evaluation efforts are focused on health programming and policy. Together we are working on revamping the Knowledge Uptake and Utilization Tool (KUUT).
Knowledge translation and exchange (KTE) have become integral parts of organizational practice, whereby knowledge generated through research and other activities is synthesized and disseminated to relevant stakeholders. As KTE has gained prominence in many disciplines, organizations are increasingly monitoring and evaluating their knowledge products and processes (e.g. reports, workshops and training activities). Various tools are being used to assess the effectiveness and impact of KTE products/processes, however, these measures are as varied as the projects, and are often not designed to evaluate KTE in particular.
In 2007, I developed the 44-item KUUT informed by KTE theories. Over the past decade, the KUUT has been used by dozens of health and non-health organizations. Taking recent literature, user and expert feedback into consideration, the KUUT is currently being re-developed and validated. This tool will aid in KTE evaluation in numerous settings, including but not limited to, health research, program and policy development.
Lessons Learned:
After conducting an extensive scoping review, we found a variety of measures being used to assess KTE and knowledge product effectiveness and impact, ranging from short-term feedback of stakeholder awareness and use, to long-term assessments of knowledge implementation and adoption. However, we found very few tools could be used across disciplines or for different knowledge product types.
Hot Tip:
As evaluators, we are often focused on assessing a broader program or strategy, however we fail to assess the products of the evaluation itself! Given the amount of effort, time, and resources that go into generating evaluation findings and KTE products like organizational reports, it is important for us to see how, and even if, our stakeholders are using these results. This exercise can help organizations tailor information for optimal knowledge uptake and use.
Rad Resources:
Though the KUUT is undergoing revisions, it is a great resource to start thinking about how to assess knowledge uptake and utilization in your work. Other assessment tools can be found in our presentation of the scoping review from the Performance and Planning Exchange Symposium in 2019. If you would like a copy of our scoping review and/or the latest version of the KUUT once it’s ready, send me an email at kskinner@uwaterloo.ca to be added to our growing Community of Practice!
The American Evaluation Association is celebrating Translational Research Evaluation (TRE) TIG week. All posts this week are contributed by members of the TRE Topical Interest Group. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Hi Ms. Skinner,
First of all, thank you for sharing a bit about your process and your KUUT. I am intrigued by your mission and am inspired to inquire further into how this tool could be used in education to support policy development. I also wonder, with some adaptation I think it would be possible for teachers who intend on making changes within their approach to teaching and learning to use a tool like this to ensure that they have effectively evaluated the impact that the intended change could have on their practice. This could serve as a helpful self-reflection tool when initiating change in order to ensure that there is sufficient understanding before proceeding.
When you were developing the KUUT what groups of stakeholders did you include in your program evaluation? To what extent did your stakeholders impact the outcome of your tool? With the development of such a tool for research, program and policy development, would you recommend taking a participation/ U-FE approach to evaluation?