Tech TIG Week: Why Evaluators (and Everyone Else) Need a New Way of Talking about AI by Paul Duignan

Hi, I am Paul Duignan, a psychologist with a background in evaluation, outcomes and strategy work, including evaluating the opportunities and risks of new technology. I currently work on artificial intelligence (AI) strategies for organizations, AI’s social and psychological impact, and new, more appropriate language for talking about AI.

The new terms being used regarding AI tend to be at a technological level—for instance, generative AI, machine learning and large language models. While understandable, we also need new conceptual language to discuss AI. Going on a road trip, most people wouldn’t talk about drive shafts, gearboxes and wheel alignment. They would discuss GPS apps, fueling stations and accommodation. What terms would be the most useful on this road trip that we are going on with AI?

Lessons Learned

Here are some new terms to help people such as evaluators and others in the organizational, policy and social sciences to discuss AI at a conceptual level.

Presentability. AI’s ability to enhance presentability means that individuals, initiatives or organizations can present themselves in their most favorable light. Given that evaluation is all about examining whether claims about an initiative are accurate, AI presentability will cause problems for many evaluation, quality assurance and regulatory systems. Any initiative, regardless of quality, will now be able to provide near-perfect documentation. In product and service evaluation, AI will be able to fabricate vast amounts of fictitious positive feedback.

Insightability. AI’s insightability means it can rapidly extract insights from large quantities of data. AI’s insightability will also potentially allow ongoing real-time evaluation rather than the more ‘snapshot’ approach currently employed in many evaluations.

Forecastability. Another game-changing feature is AI’s forecastability—its ability to use simulations to identify what is likely to occur. AI’s simulation potential will allow a reinvention of ex-ante evaluation (undertaken before an intervention).

Trustability. A further question is how people will react to the AI-induced trustability crisis. Will AI drive many people into firewalled communities for protection against fake information (a tsunami of infotrash) and fake identities.

AI-ready society. At a more societal level we should be talking about the AI-ready society. It’s not just AI’s features that will determine its impact on society so much as how ready a particular society is to receive it. Another important issue is how the AI dividend—the wealth likely to be created by AI—will be distributed. We should also be considering how to have healthy relationships with AI buddies. AI-withdrawal syndrome may arise when people leave their AI-generated perfectopia worlds and hunger for reality.Where some people may seek out non-AI experiences and non-AI-infused goods and services as a reaction to an AI-saturated world.

Rad Resources

• The AI Daily Brief is good for quick updates on where AI is going.
• Dailyai.com provides some in-depth articles.
• This 12 AI podcasts article lists relevant podcasts.
• My substack and podcast DrPaulNewIdeas looks at AI from a strategic and social impact point of view and includes a six-minute video on where AI is at the moment you can use in any presentations on AI and evaluation.
• To learn more about these terms here’s my new book Surfing AI: 30 Fresh Terms and Smarter Ways of Talking About Artificial Intelligence.

---

The American Evaluation Association is hosting Integrating Technology into Evaluation TIG Week with our colleagues in the Integrating Technology into Evaluation Topical Interest Group. The contributions all this week to AEA365 come from ITE TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.