My name is Molly Linabarger and I am a part of the Evaluation and Research for Action practice at Deloitte Consulting LLP. I serve as an evaluator for government clients, helping them to assess, understand, and communicate the implementation and impact of their programs.
You likely have heard about Generative Artificial Intelligence, or GenAI. Precursor forms of AI were designed to analyze data and make predictions on that data. So, what’s all the excitement around GenAI? GenAI can do all that AI can do and more: GenAI can create new data.
GenAI is being used across the field of evaluation in a variety of ways, from data cleaning, analysis, and reporting/dissemination, and has the potential to change the landscape of evaluation. But not all uses have to be complex or in-depth – I’m here to share a few simple tips and tricks to begin using GenAI in your daily evaluation practice.
Hot Tips:
Get smart on GenAI prompts: We all know that an evaluation is only as good as its questions, and same thing here! Prompts are the starting point for GenAI – they tell the tool what you want it to do. Practice writing concise, clear prompts; hints on how to start can be found in the following examples.
- Get your juices flowing. Need to create an evaluation tool and not sure where to start? Ask a GenAI tool to generate an example with a prompt like “Generate an interview guide for [population] that answers [these evaluation questions]” and use the results to continue your brainstorming process! The more detail you can include in your prompt, the better.
- Adjust the reading level of your writing. Unsure how to de-jargon your language to be more appropriate for a specific audience? Ask a GenAI tool to “Rewrite this for a [specific grade level]” and take cues as to how you can edit your language.
- Check your writing for clarity. Not sure if you’re communicating clearly enough? Flip the script on GenAI and ask a tool to summarize your writing: “Summarize the key findings in 2-3 bullets” and check how well that aligns with your intention.
- Use GenAI for mundane tasks. Have a list that you need to sort alphabetically, by word count, or frequency? Ask a GenAI tool to “Sort this list alphabetically by the first word.” You can even ask it to include or remove bullets or extra lines between items in the list – play around with it!
As with any tool, you want to be aware of the associated limitations, and GenAI isn’t without limitations.
- You must consider if GenAI is an appropriate tool for your work – check the requirements and guidelines for your work. If you’re not sure, ask your clients, partners, and other stakeholders whether this is an appropriate use.
- Remember that GenAI is only as good as the data fed into it – ask yourself questions around the representativeness of the data and relevance to your evaluation.
- Carefully consider the data that you are putting into a GenAI tool and remember the importance of protecting privacy and confidentiality. Open-source AIs can use data entered by users; a general rule of thumb is to not share anything you wouldn’t publish.
While these are simple tips that can help you incorporate GenAI into your daily evaluation practice, we should continue to be critical about how large-scale applications of GenAI will change evaluation practice. Learn more about what that might look like in this AEA 365 blog post.
The American Evaluation Association is hosting Health Evaluation TIG Week with our colleagues in the Health Evaluation Topical Interest Group. The contributions all this week to AEA365 come from our Health Evaluation TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.