Hello, AEA365 community! Liz DiLuzio here, Lead Curator of the blog. This post is a continuation from Transformational Eval Week (12/10/23-12/16/23), a topic so hearty that we couldn’t contain the posts to just seven. Please enjoy.
I’m Mita Marra. As an economics professor at the University of Naples in Italy, specialized in policy evaluation in regional development, innovation and work-family interface, my engagement with the evaluation community has spanned over two decades. I have served as the Editor-in-Chief of the international journal Evaluation and Program Planning since 2019, and as President of the Italian Evaluation Association (AIV) between 2013 and 2017. Currently, I am a member of the Board of the European Evaluation Society and the Council of the International Evaluation Academy.
Here, I intend to share some reflections on decolonizing evaluation, addressing efficiency-related concerns evident in two seemingly unrelated studies I conducted recently.
The first study examined the collaborative exhibition-making efforts of the National Archaeological Museum in Naples, Italy. The Museum strategically expanded its network of partners and stakeholders, employing innovative technologies and games to engage a younger audience in person and online, especially during the pandemic.
The second study scrutinized how a well-established tech company in Italy designed a data infrastructure for machine learning to provide information solutions. Leveraging its proprietary dataset, the company aimed to assess the digital upgrading required by potential clients to tailor sellable service packages.
Despite their distinct objectives, these studies converged on a shared theme: the identification of “slack,” referring to underutilized resources with the potential to affect organizational capacity to adapt and innovate. Drawing inspiration
from Hirschman’s theory of Exit, Voice, and Loyalty, both evaluations proposed strategies to counteract deterioration and enhance value creation process, strengthening organizational capacity to navigate diverse situations.
In the context of the National Archaeological Museum, evaluators unveiled a slack in collaborative exhibition-making—a political capability that empowered local actors but risked weakening the self-funding mechanism for cultural offerings. The delicate balance between inclusiveness and financial autonomy prompted the Museum to assess the sustainability of its partnerships—drawing on the newly generated metrics on unbalanced costs and risks shared in exhibition co-production, their duration, and content innovativeness.
With the tech company, the limited availability of Key Performance Indicators (KPIs) led data managers to overlook firm diverse needs across sectors and regions. Identifying this slack, evaluators recommended an expansion of information at the firm and regional levels, compelling data managers to incorporate the variability of business models and performance in their AI development.
The concept of slack served as a lens through which organizational ineffciencies were identified and addressed. Both evaluations were transformational in that they led to the utilization of new and untapped first-hand data to reshape strategies and broaden organizational offerings.
So going back to my original effciency-related concerns, did these evaluations contribute to decolonization? Indeed, they did.
Lessons Learned
Both evaluations challenged the effciency-centric notion of the best way of doing things relying solely on quantified inputs and outputs. Evaluators unearthed equity-based insights into promising partnerships with marginalized artists in contemporary and pop arts or the needs of firms in less innovative and less developed sectors in the region. By exploring hidden informational resources within and beyond the organization, evaluators voiced a commitment to decolonizing evaluation. It goes beyond the conventional approach based on gauging revenue- and visitors-based exhibition success or verifying universal technology standards for businesses with parsimonious information fed into an AI algorithm. Decolonization in these studies signified enriching and integrating the evaluative informational base through dialogue with workers, partners, and customers of the organizations involved.
By shifting to an outside-in perspective, and making sense of highly heterogeneous information, future evaluations can eschew a compliance mentality and an extractive data and measurement practice, thereby enhancing learning and empowering local change agents.
References
Marra, M., Celentano, R., Covino, R. (2023) Coproduction and Self-Financing for Local Development: The Exhibitions of the National Archaeological Museum of Naples, Italian Journal Regional Science, forthcoming.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.