OL-ECB Week: Jeff Sheldon on using ROLE to Determine an Organization’s Support of Evaluative Inquiry

I’m Jeff Sheldon from Claremont Graduate University and today I’m sharing the Readiness for Organizational Learning and Evaluation (ROLE) instrument because I think it’s a vastly under-utilized resource that evaluation practitioners should consider when they want to find out whether an organization is a learning organization or ready for an evaluation.

In my own research I used ROLE to determine whether elementary schools would likely use evaluative inquiry as a basis for decision-making if there were certain organizational learning contextual variables in place.  One of the things I wanted to investigate was the adaptability of ROLE in a context (elementary schools), and with a population (elementary school principals, administrators, faculty, and staff), that have not been widely investigated a propos of organizational learning and evaluation and show ROLE’s efficacy as the sole quantitative measure of organizational learning and evaluation in an empirical study.  However, one of the great things about the ROLE is that it can be adapted for use across a wide spectrum of organizational types and sectors, but I offer elementary schools as one example of its use.

For those of you who don’t know, the ROLE is a highly reliable (Cronbach’s Alpha = .97) instrument developed by Preskill and Torres in 2000 to operationalize the constructs of organizational learning – culture, leadership, systems and structures, communication, and teams along with an evaluation construct.  These constructs constitute the internal elements, or as I call them, contextual variables, that must be in place if an organization is to support and encourage organizational learning and evaluative inquiry.  Unlike other quantitative instruments, the ROLE measures organizational learning constructs that are concrete and well-established in the extant literature, reflects the multiple dimensions of organizational learning, and was designed to help you determine an organization’s level of readiness for implementing organizational learning, evaluation practices, and their supporting processes.

This is a brief “snap-shot” of one section of the ROLE as adapted for my study with schools.  The item tag was the same as on the original ROLE, but I changed it from “We” to “Faculty & Staff” to reflect school terminology.

Hot Tip:  The results from the ROLE can be used to: identify the existence of learning organization characteristics, diagnose interest in conducting evaluation that facilitates organizational learning, identify areas of strength to leverage evaluative inquiry processes, and identify areas in need of organizational change and development.

Rad Resource:  If you’re looking for the ROLE, look no farther: Preskill, H., & Torres, R. (2001). The readiness for organizational learning and evaluation instrument (ROLE). In D. Russ-Eft & H. Preskill (Eds.), Evaluation in organizations: A systematic approach to enhancing learning, performance, and change. New York: Basic Books.

The American Evaluation Association is celebrating Organizational Learning & Evaluation Capacity Building (OL-ECB) TIG Week with our colleagues in the OL-ECB AEA Topical Interest Group. The contributions all this week to aea365 come from our OL-ECB TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.