We are Shannon L. Griswold, Ph.D., a scientific research evaluator and member of AEA’s Research Technology and Development TIG, Alexandra Medina-Borja, Ph.D., Associate Professor of Industrial Engineering at University of Puerto Rico-Mayaguez, and Kostas Triantis, Ph.D., Professor of Systems Engineering at Virginia Tech. We are thinking about new ways to envision and evaluate impacts from discovery-based scientific research. Tracing dollars spent on funding research in universities to societal impacts is very difficult due to the long time lag between experimentation and commercialization, and the serendipitous nature of discovery.
Lesson Learned: Even though we can’t predict every outcome of scientific research, we can apply a general framework that allows us to envision the complex system of scientific discovery and identify areas of inquiry that could lead to major breakthroughs.
Hot Tip: Gather your research community and ask them to think backwards from societal needs (e.g., in transportation research this might be a solution for traffic congestion). This can be HARD for fundamental researchers; they are accustomed to letting curiosity drive their research questions. From societal needs, ask them to map several enabling technologies that could meet that need. Enabling technologies should be things that could solve that need but that don’t exist yet (e.g., teleportation). Finally, from enabling technologies, ask your research community to map out knowledge gaps. These are the things that we don’t know yet, which prevent us from developing enabling technologies (e.g., how do you convert all the mass in a human body into energy without blowing things up? How do you reassemble that energy at the destination into a human body?). It can be helpful to frame knowledge gaps as questions.
Hot Tip: Use societal needs, enabling technologies, and knowledge gaps to perform a content analysis of your research portfolio. How many of the topics are already funded? How many topics are not yet represented in the portfolio? This analysis should be performed in the context of a portfolio framework, which may help you envision the scope of your funding program’s discipline and relation to other funding streams.
Rad Resource: When mapping societal needs, enabling technologies, and knowledge gaps, it can be helpful to place them in a hierarchical framework to track their relationships. In this diagram, dotted lines show the direction in which the logic framework is generated, working backwards from societal needs. Solid arrows show the flow of scientific knowledge, from discoveries (knowledge gaps) to technologies that meet societal needs.
Rad Resource: The flow of knowledge and information in the scientific process is rarely linear. It is probably more accurately represented as a “ripple effect”. We can predict some discoveries and technologies (darker polygons), but others are emergent, and knowledge flows in all directions.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.