I’m Chithra Adams, a program evaluator at the Human Development Institute, University of Kentucky. I explore how design principles and research can be applied to program evaluation.
Rad Idea – prototyping: A key part of the design process is prototyping. Prototyping is a process by which ideas and concepts are tested for technical and social feasibility. Prototypes are physical products. They can range from the expression of an early idea to an almost complete product. Depending on the context, prototyping can be used in several ways (Stapper, 2010):
Prototypes evoke a focused discussion in a team because the phenomenon is ‘on the table’.
Prototypes allow testing of a hypothesis.
Prototypes confront theories because instantiating one typically forces those involved to consider several overlapping perspectives/theories/frames.
Prototypes confront the world because the theory is not hidden in abstraction.
A prototype can change the world because in interventions it allows people to experience a situation that did not exist before.
Lessons learned in prototyping evaluation products and process: I use prototyping to test out new ideas as well as to get the client involved in the evaluation process. Prototyping can be used both to test out ideas for products (reports, briefs) and process (client interaction, stakeholder involvement). Regardless, the client/stakeholder should understand that the prototype is only a draft and further refinements will be made. In some cases, prototyping with a client/stakeholder is simply not feasible. In those cases, the idea or product can be tested with someone outside a project. Sometimes I try out the first few iterations with other evaluators and test the final versions with clients.
The timing of the prototyping process is critical. It should be done when the client has the time to provide feedback for further refinement. You cannot prototype an evaluation report a week before it is due! The key to prototyping is rapidly testing ideas and getting feedback. Be receptive to feedback. Being receptive to it does not mean that all feedback will be incorporated in the next iteration. Expert judgment should be used to identify what feedback will be adopted.
Rad Resources:
Sanders and Stapper (2014) describe various processes designers use to gain insights and test ideas with consumers.
While this blogpost describes prototype within the context of web development, it provides a really broad snapshot about the nuts and bolts of prototyping.
The American Evaluation Association is celebrating Program Design TIG Week with our colleagues in the Program Design Topical Interest Group. The contributions all this week to aea365 come from our Program Design TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.