Hi! I’m Madeleine Pope (they/them), research associate and project manager at HG&Co, a visitor-centered planning, strategy, & evaluation firm which supports museums and cultural organizations.
Having just attended my first AEA conference as a new museum evaluator, I noticed that there are some differences between my niche world of work and organizational or teaching evaluation. I’ve noticed we use different lingo for evaluation phases, not saying product or implementation evaluation, but rather front-end (including needs assessments), formative (including usability), remedial (iterative changes), and summative (impact and goal achievement) evaluations. Clients ask us questions like, “What do we need to know about our locals before writing a new strategic plan?”, “How can we make this software useful for both planetarium managers and space hobbyists?”, and “Does this game about transcendentalism help teachers teach Thoreau?”
Another difference, perhaps, is the push to hire non-evaluators to do evaluation.
The Visitor Studies Association (VSA)’s recent call for proposals summarizes this trend. Roughly: given museums’ histories of exclusivity in favor of wealth and whiteness, ethical evaluations for these institutions must include attempts to broaden the viewpoints of decision makers. It is our responsibility to bring in non-white and non-wealthy perspectives, especially as many board members and directors identify as the opposite. But many evaluators also have backgrounds of privilege, at least a privileged education if not a privileged race as well. (To note: I am white and come from a lower/middle class background).
We know that the evaluative process will have some subjectivity, even with best practices duly prioritized. A white data collector (often me) interviewing museum visitors of color may receive different responses than an evaluator of the respondent’s own race or color, especially if sensitive questions are being asked about identity or money. Recognizing the racial and wealth inequities of who is allowed into higher education, we benefit from welcoming in non-evaluators into the work. The VSA summarizes: “By collaborating with diverse stakeholders, we can elevate these unique viewpoints and benefit from new knowledge and perspectives.”
I am fortunate to have experienced this. Our firm, HG&Co, hired and trained a local Haitian-American teaching artist to collect data in Haitian communities around our art museum client, which was lacking Haitian visitors despite close range. The experience was deeply educational; we would not have gotten data that high in quality without her.
That project, as many others for museums, focused on meaning-making, perhaps a more iterative goal than tracking use of a new website or amount of time spent with an exhibit. The art museum wanted support changing its identity with the neighbors; qualitative data exploring ideas outside the walls were critical. This type of data collection requires some softer people skillsthat aren’t requisite for trained evaluators. Understanding racialized, economic, and local experiences, let alone the local language (Haitian Creole), was necessary for the data we needed. As my coworker put it: You’re not gonna get new, rich data by not trying new ways of data collection.
I wonder how this trend would affect evaluation outside of museums, arts, and cultural organizations. What would change about your regular processes if you hired outside the field? How do you compare lived experiences with traditional education? What do evaluators lose by requiring higher education degrees for entry level positions?
The American Evaluation Association is hosting Arts, Culture, and Museums (ACM) TIG Week. The contributions all week come from ACA TIG members. Do you have questions, concerns, kudos, or content to extend this AEA365 contribution? Please add them in the comments section for this post on the AEA365 webpage so that we may enrich our community of practice. Would you like to submit an AEA365 Tip? Please send a note of interest to AEA365@eval.org. AEA365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators. The views and opinions expressed on the AEA365 blog are solely those of the original authors and other contributors. These views and opinions do not necessarily represent those of the American Evaluation Association, and/or any/all contributors to this site.