CAT | Organizational Learning and Evaluation Capacity Building
15
Karen Widmer on Knowledge Flow: Making Evaluation the Reference Point
4 Comments · Posted by Sheila Robinson in Organizational Learning and Evaluation Capacity Building
Hello. I am Karen Widmer, a 4th year doctoral student in the Evaluation program at Claremont Graduate University. I’ve been developing and evaluating systems for performance (business, education, healthcare, and nonprofits) for a long time. I think organizations are a lot like organisms. While each organization is unique, certain conditions help them all grow. I get enthusiastic about designing evaluations that optimize those conditions!
Theme: My master’s research project looked at evaluation-related activities shared by high-performing organizations. For these organizations, evaluation was tied to decision making. Evaluation activity pulled together knowledge about organizational impact, direction, processes, and developments, and this fed the decisions. The challenge for evaluation is to pool the streams of organizational knowledge most relevant for each decision.
Hot Tip:
- Evaluative thinking identifies the flow of organizational knowledge and this provides decision makers with a point of reference for quality decisions.
- In technical language, Knowledge Flow may mediate or moderate the relationship between evaluative thinking and decision quality. Moreover, the quality of the decision could be measured by the performance outcomes resulting from the decision!
Cool Trick:
- Design your evaluation to follow the flow of knowledge throughout the evaluand lifecycle.
- Document what was learned when tacit knowledge was elicited; when knowledge was discovered, captured, shared, or applied; and knowledge regarding the status quo was challenged. (To explore further, look to the work of: M. Polanyi, I. Becerra-Fernandez, and C. Argyris and D. Schon.)
- For the organizations I looked at, these knowledge activities contained the evaluative feedback desired by decision makers. The knowledge generated at these points told what’s going on.
- For example, tacit perceptions could be drawn out through peer mentoring or a survey; knowledge captured on a flipchart or by software; or a team might “discover” knowledge new to the group or challenge knowledge previously undisputed.
Conclusion: By design or still shot, evaluative thinking can view the flow of knowledge critical to decisions about outcomes. Knowledge Flow offers a framework for connecting evaluation with the insights decision makers want for reflection and adaptive response. Let’s talk about it!
Rad Resource: The Criteria for Performance Excellence is a great government publication that links evaluative thinking so closely with decisions about outcomes that you can’t pry them apart.
Rad resource: Neat quote by Nielsen, Lemire, and Skov in the American Journal of Evaluation (2011) defines evaluation capacity as “…an organization’s ability to bring about, align, and sustain its objectives, structure, processes, culture, human capital, and technology to produce evaluative knowledge [emphasis added] that informs on-going practices and decision-making to improve organizational effectiveness.”
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Capacity Building · evaluative thinking · knowledge flow · organizational learning
7
Johanna Morariu, Kat Athanasiades, and Ann Emery on New Research: The State of Evaluation 2012
5 Comments · Posted by Sheila Robinson in Nonprofits and Foundations Evaluation, Organizational Learning and Evaluation Capacity Building, Qualitative Methods, Quantitative Methods: Theory and Design, Research on Evaluation
Hello! We are Johanna Morariu, Kat Athanasiades, and Ann Emery from Innovation Network. For 20 years, Innovation Network has helped nonprofits and foundations evaluate and learn from their work.
In 2010, Innovation Network set out to answer a question that was previously unaddressed in the evaluation field—what is the state of nonprofit evaluation practice and capacity?—and initiated the first iteration of the State of Evaluation project. In 2012 we launched the second installment of the State of Evaluation project. A total of 546 representatives of 501(c)3 nonprofit organizations nationwide responded to our 2012 survey.
Lessons Learned–So what’s the state of evaluation among nonprofits? Here are the top ten highlights from our research:
1. 90% of nonprofits evaluated some part of their work in the past year. However, only 28% of nonprofits exhibit what we feel are promising capacities and behaviors to meaningfully engage in evaluation.
2. The use of qualitative practices (e.g. case studies, focus groups, and interviews—used by fewer than 50% of organizations) has increased, though quantitative practices (e.g. compiling statistics, feedback forms, and internal tracking forms—used by more than 50% of organizations) still reign supreme.
3. 18% of nonprofits had a full-time employee dedicated to evaluation.
4. Organizations were positive about working with external evaluators: 69% rated the experience as excellent or good.
5. 100% of organizations that engaged in evaluation used their findings.
6. Large and small organizations faced different barriers to evaluation: 28% of large organizations named “funders asking you to report on the wrong data” as a barrier, compared to 12% overall.
7. 82% of nonprofits believe that discussing evaluation results with funders is useful.
8. 10% of nonprofits felt that you don’t need evaluation to know that your organization’s approach is working.
9. Evaluation is a low priority among nonprofits: it was ranked second to last in a list of 10 priorities, only coming ahead of research.
10. Among both funders and nonprofits, the primary audience of evaluation results is internal: for nonprofits, it is the CEO/ED/management, and for funders, it is the Board of Directors.
Rad Resource—The State of Evaluation 2010 and 2012 reports are available online at for your reading pleasure.
Rad Resource—What are evaluators saying about the State of Evaluation 2012 data? Look no further! You can see examples here by Matt Forti and Tom Kelly.
Rad Resource—Measuring evaluation in the social sector: Check out the Center for Effective Philanthropy’s 2012 Room for Improvement and New Philanthropy Capital’s 2012 Making an Impact.
Hot Tip—Want to discuss the State of Evaluation? Leave a comment below, or tweet us (@InnoNet_Eval) using #SOE2012!
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
Capacity Building · Data · Evaluation · nonprofit · organizational learning · reports
11
Bloggers Series: Brian Hoessler on the Strong Roots Blog
No comments · Posted by Susan Kistler in Nonprofits and Foundations Evaluation, Organizational Learning and Evaluation Capacity Building
Hi everyone, my name is Brian Hoessler and I am an independent consultant in Saskatoon, Canada. Through my company, Strong Roots Consulting, I work with non-profit organizations and community-based initiatives to build their capacity through research, strategic planning, and evaluation.
Rad Resource – Strong Roots blog: The website for Strong Roots is also home to my professional blog, which I use to share ideas, resources, and news with the non-profit community in and beyond Saskatoon. It also provides a behind-the-scenes look at the ongoing development of my consulting business, having just started it this past July.
Lessons Learned – why I blog: My reasons for blogging include a bit of everything – sharing tips and resources, demonstrating my knowledge and skills to a new community (I’ve been in Saskatoon for less than a year), and supporting my own professional development. As someone new to the consulting field who is just becoming comfortable with the term “evaluator”, I also use my blog as a space to reflect on my practice and think about directions to take.
Hot Tips – favorite posts:
- When Does It End? – This post demonstrates how connecting with an online community of bloggers can lead to fruitful thinking. A post by Chris Lysy at freshspectrum (via EvalCentral) prompted me to write about how a dose of evaluative thinking can help determine when a program is failing, even if things look good on the surface.
- En Route – A demonstration of how I think through writing, in this case a reflection on the term “evaluator” and how I identify (or not) with the field.
- AEA Conference Day 1 – When I attended my first AEA conference this past October in Minneapolis, I decided to post daily to share new ideas, resources, and personal insights. It was sometimes difficult to find the energy to write after a long day of workshops and sessions, but I’m glad that I kept it up!
Lessons Learned – what I’ve learned: I’ve found it useful to keep a couple of drafts or expanded outlines at hand – sometimes I’ll come up with an idea for a post but don’t feel like writing it out right then and there, or I’m in the mood for stringing words together but have nothing pressing to write about. Breaking the blogging process into two parts, idea generation and writing, can help lessen the anxiety of seeing that blank page!
This winter, we’re continuing our series highlighting evaluators who blog. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
No tags
20
Wendy Tackett and Joseph Trommater on Local Evaluation Capacity Building
1 Comment · Posted by jgothberg in Independent Consulting, Organizational Learning and Evaluation Capacity Building, Prek-12 Educational Evaluation
Hi! I’m Wendy Tackett, President of iEval, an evaluation consulting firm in Michigan, and I’m Joseph Trommater, Project Director for S.P.A.R.K.S., a 21st Century Community Learning Centers (CCLCs) program funded in northern Michigan through the Michigan Department of Education. iEval serves as the external evaluators for S.P.A.R.K.S. and has maintained that relationship for ten years. The S.P.A.R.K.S. program operates with the belief that evaluation should be part of the way we do business and drives change. In fact, when the original Project Director was hired, she was given Wendy’s business card as a starting point. Carpe diem!
Lesson Learned: When you build evaluation into the process from the very beginning, it becomes a necessary part of the program instead of an “add on” later. Staff and clients see the integration of evaluation as seamless, which means they’re more likely to respond to requests for data (e.g., surveys, interviews, focus groups) and be open and honest in their sharing. When the evaluation team for 21st CCLCs conducted their first statewide surveys of afterschool staff, several questions asked about staff knowledge and use of evaluation. Our program sites scored low compared to the rest of the state. After digging into that, we found that it was because the staff didn’t think of the data and use of data as evaluation – they just saw it as an important part of their work. Results from the intentional focus on using data show improvements in student attendance, behavior, and academic achievement.
Hot Tip: Instead of sharing evaluation data in huge reports then walking away, we take the time to carefully plan using the data. We receive three primary evaluation reports throughout the year from iEval as well as supplemental evaluation data from the state evaluation and technical assistance teams. That’s a lot of data! We take the time to go through the reports when we first get them, but we’ve also created a data calendar for the year. Each month we guide staff to operationalize the data around a specific theme (e.g., student recruitment, student retention, academic achievement), and we identify where in each report they can find supporting data for that theme. We discuss it at staff meetings and create small goals that can be met each month that will lead to our overall goal of students being successful in school. The use of data in improving programs is always on our staff meeting agendas!
Rad Resources:
- “21st Century Community Learning Centers Local Evaluator Guide – Second Edition” by Michigan’s Local Evaluator Advisory Committee gives tips for working with your local evaluator in your after school programs.
- The “Evaluation Exchange” through the Harvard Family Research Project has great information, including a section on participatory evaluation.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
17
Annaliese Calhoun on Measuring Sustainability Capacity and Planning for Long Term Success
6 Comments · Posted by jgothberg in Organizational Learning and Evaluation Capacity Building
Hi, I’m Annaliese Calhoun, Project Manager at the Center for Public Health Systems Science at Washington University in St. Louis. My team has been developing the Program Sustainability Assessment Tool, a new tool designed to help programs to rate their sustainability capacity. We define sustainability capacity as the ability to maintain programming and its benefits over time.
After testing reliability and refining the tool, the 40-item Program Sustainability Assessment Tool is now available in a free online format along with other great resources on sustainability planning.
Lessons Learned: Program sustainability capacity is about more than just funding. Through our research, we’ve identified eight key domains that influence a program’s ability to continue providing services and their benefits over time. One of these key components is program evaluation. The full list includes:
- Political Support: Internal and external environments that support your program
- Funding Stability: Establishing a consistent financial base for your program
- Partnerships: Cultivating connections between your program and its stakeholders.
- Organizational Capacity: Having the internal support and resources needed to effectively manage your program
- Program Evaluation: Assessing your program to inform planning and document results
- Program Adaptation: Taking actions that adapt your program to ensure its ongoing effectiveness.
- Communications: Strategic communication with stakeholders and the public about your program.
- Strategic Planning: Using processes that guide your program’s directions, goals, and strategies.
- Building program sustainability capacity requires assessment and planning. The Program Sustainability Assessment Tool was designed to identify a program’s areas of sustainability strength and challenge. Program staff and stakeholders can then use results from this assessment to inform sustainability planning.
Rad Resource: I’m excited to announce the launch of this new website www.sustaintool.org featuring the Program Sustainability Assessment Tool and sustainability planning resources. You can use this site to:
- Deepen your understanding of the factors affecting sustainability capacity.
- Create a free account and assess your program’s sustainability capacity. When you’re finished, you’ll get a summary sustainability report.
- Invite other people to assess the same program and have all the responses totaled in a group summary sustainability report.
- Explore tools and resources for developing a program sustainability plan.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
16
OL-ECB Week: Jesse Burns on Being an Independent Consultant
No comments · Posted by sgrant in Organizational Learning and Evaluation Capacity Building
Greetings! My name is Jesse Burns, and I am part of the OL-ECB TIG leadership. Outside of AEA, I am an independent consultant that works with a variety of organizations to design, implement and evaluate new products and services.
As an independent consultant, I have learned that managing the uncertainty with innovation projects is critical for gaining internal support. While I received training with developing rigorous formative and summative evaluations, I’ve noticed that trying to inject these methods into uncertain innovation projects can lead to push back from the client.
In working on projects where clients were not interested in applying traditional evaluation methods to a project, I learned that clients often wanted to first understand the implications of a new innovation and then understand how they could evaluate the impacts of the innovation. With this recognition in mind, I started applying insights from work by Amy Edmondson about the different processes that individuals and teams face with learning:
Hot Tips:
Addressing the different types of learning challenges can help different stakeholders to understand and engage in a project that requires them to learn.
Team Challenge Activities: Rather than talking about learning, engaging individuals, teams and units in a puzzle or game can help illustrate how these different challenges manifest in individuals and teams. Using a game or puzzle to highlight and discuss these challenges can help surface the tensions between individual and organizational learning.
Problem Framing Activities: As a consultant, part of my work involves framing the problem that a client has to help them see their options a little more clearly. On innovation projects, engaging clients in framing their problem via Force-Field Analysis, SWOT analysis, or other visually-based activities can not only help frame the problem, but also uncover the learning challenges that individuals and teams will face in addressing the problem.
Rad Resources:
Written by Preskill and Beer, this a great report on how to use developmental evaluation methods to evaluate social innovation projects. They provide a great overview of the limitations of using traditional evaluative methods to evaluate innovation projects as well as outline how developmental evaluation might be the right approach in these instances.
Written by Garvin, Edmondson and Gino, this classic 2008 HBR article teases apart the challenges of learning in organizations, as well as provides a great tool for assessing whether your organization (or the organization you are consulting with) is a learning organization.
The American Evaluation Association is celebrating Organizational Learning & Evaluation Capacity Building (OL-ECB) TIG Week with our colleagues in the OL-ECB AEA Topical Interest Group. The contributions all this week to aea365 come from our OL-ECB TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
15
OL-ECB Week: Bonnie Richards on Setting the Stage: Evaluation Preparation and Stakeholder Buy-In
No comments · Posted by sgrant in Organizational Learning and Evaluation Capacity Building
My name is Bonnie Richards, and as a professional with both experience and an academic background in Evaluation and Organizational Behavior, I have had the opportunity to facilitate the evaluation and learning process with stakeholders in many different industries.
Although my experiences have been unique across projects, I have noticed a common thread that ties them all together -besides evaluation, of course. Generating buy-in from stakeholders at the outset defines the entire evaluation process, and ultimately impacts the utility and sustainability of the project.
The following are some important steps that I use to create buy in with stakeholders and start the evaluation process off right.
Hot Tips:
1. Start by getting to know the program/process/organization better. For example, learn why an evaluation is being conducted.
- Was evaluation required by a funder or stipulated in a grant? Has the organization secured funding because it is intrinsically motivated to learn?
2. Explain what evaluation is, and what it is not.
- The evaluator is not there to question the intentions or efforts of staff.
- The evaluator is there to help answer some important questions that can help an organization learn what it is doing well, and what areas exist for potential improvement.
- How do stakeholders feel about evaluation? Check for evaluation baggage and anxiety, and ask stakeholders to share what the word “evaluation” means to them.
3. Create open communication from the beginning, and include different stakeholder groups. Consider how you can include stakeholders in meaningful and engaging ways.
- What do stakeholders want to learn? What questions are most important for them to answer at this point in time?
- Stakeholders are experts in their content areas. Don’t forget to solicit and integrate their feedback.
By preparing stakeholders for an evaluation you increase the chance that they will be more comfortable and willing to ask questions and contribute throughout the process. This not only benefits the current evaluation, but can also increase their confidence and competence when entering into evaluation experiences in the future –and this also has the added benefit of making our lives as evaluators a bit easier the next time around.
Rad Resource:
Because evaluators work with stakeholders from various backgrounds and perspectives, it is important to think about how we work with these different groups. Adult Learning Basics by William Rothwell provides a theoretical and practical guide to understanding and engaging stakeholders in meaningful interactions.
The American Evaluation Association is celebrating Organizational Learning & Evaluation Capacity Building (OL-ECB) TIG Week with our colleagues in the OL-ECB AEA Topical Interest Group. The contributions all this week to aea365 come from our OL-ECB TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
14
OL-ECB Week: Joseph E. Bauer on Being an Internal Evaluator
No comments · Posted by sgrant in Organizational Learning and Evaluation Capacity Building
Hi, I’m Joe Bauer, the director of Survey Research in the Statistics & Evaluation Center at the American Cancer Society – National Home Office in Atlanta, Georgia. I have been working as an internal evaluator for almost seven years, in one of the most dynamic, challenging, and rewarding positions that I have ever had.
Lesson Learned:
Remember who the focus is. I came to the American Cancer Society to have an impact at a national level. Each and every day my focus is on doing everything I can to benefit cancer patients, their caregivers, and their family. Using that as my rubric helps me determine what is important enough to spend the majority of my time and effort on. In addition, our work also has a fiduciary responsibility to our donors – so my work always involves being as cost-efficient as possible.
Hot Tips:
Do nothing you cannot defend. Be a team player and always play nice in the sand box, however, do not compromise your values. This is crucially important – because you always need to do the right thing. Keep your integrity – because that is something you need to own throughout your career.
Evaluator’s adage: Life is not fair and life is not easy. Keeping your integrity will be difficult at times and it will not always be easy to do the right thing. When you least expect it – you will be challenged by ethical dilemmas. There will be no comfort, aside from the fact that many other evaluators have experienced these pressures and come out the other side. No matter how you come out on the other side – these challenges will be with you the rest of your life – they make you who you are.
Always take the high road and always push for quality. There will be others who want you to compromise on quality and take short cuts. There are always time and monetary considerations on any project – wrestle with getting as much quality into your endeavors as you can – give every client your very best – because it is your work products that define you and tell people what you stand for.
Rad Resources:
The American Evaluation Association’s Guiding Principles for Evaluators – which are intended to guide the professional practice of evaluators, and inform evaluation clients and the general public about the principles they can expect to be upheld by professional evaluators.
The National Center for Professional & Research Ethics brings together information on best practices in research, academia, and business in an on-line portal and Center. The Center is in the process of developing, gathering, preserving, and providing comprehensive access to resources related to professional and research ethics for multiple audiences.
The American Evaluation Association is celebrating Organizational Learning & Evaluation Capacity Building (OL-ECB) TIG Week with our colleagues in the OL-ECB AEA Topical Interest Group. The contributions all this week to aea365 come from our OL-ECB TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
13
OL-ECB Week: Laura Keene on Activities for Evaluation Training
No comments · Posted by sgrant in Organizational Learning and Evaluation Capacity Building
Hello! My name is Laura Keene, owner of Keene Insights, a one-woman consulting shop based in sunny Los Angeles, California. Over the past nine years, I’ve worked as both an internal and an external evaluator in a variety of settings. For many organizations, especially smaller nonprofits, working with me is their first introduction to evaluation. The success of these projects hinges on good capacity building as I work to overcome their fears and give them the tools they need to make design decisions and use findings. (See Reid Zimmerman’s post for a great description of evaluation fears).
Sometimes this work requires sessions that are solely focused on training. Other times we move back and forth between learning, discussion, and decision making. In either case, I try to integrate activities that tap into adult-learning principles and actively engage the group as much as possible. This makes the training more effective and much more fun.
Hot Tips:
In the excitement of planning great activities, don’t forget to keep these basic training principles in mind:
- Put yourself in the shoes of your participants. What is their background? What is their perspective on the topic? What do they know already? What are they expecting to gain from the training overall?
- Keep the purpose of the activity in the forefront. What’s the main point? Why does it matter? All components should support the purpose.
- Be as prepared as humanly possible. Knowing the activity inside and out gives you the flexibility to handle the unexpected and adapt where needed.
Rad Resource:
Building Evaluation Capacity by Hallie Preskill and Darlene Russ-Eft is a great source for activities and ideas. It has step-by-step instructions for activities that focus on everything from exploring the differences between evaluation and research to understanding and interpreting data. Handouts included!
For a more comprehensive training guide check out Bruce Klatt’s Ultimate Training Workshop Handbook.
The American Evaluation Association is celebrating Organizational Learning & Evaluation Capacity Building (OL-ECB) TIG Week with our colleagues in the OL-ECB AEA Topical Interest Group. The contributions all this week to aea365 come from our OL-ECB TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.
12
OL-ECB Week: Jeff Sheldon on using ROLE to Determine an Organization’s Support of Evaluative Inquiry
1 Comment · Posted by sgrant in Organizational Learning and Evaluation Capacity Building, Uncategorized
I’m Jeff Sheldon from Claremont Graduate University and today I’m sharing the Readiness for Organizational Learning and Evaluation (ROLE) instrument because I think it’s a vastly under-utilized resource that evaluation practitioners should consider when they want to find out whether an organization is a learning organization or ready for an evaluation.
In my own research I used ROLE to determine whether elementary schools would likely use evaluative inquiry as a basis for decision-making if there were certain organizational learning contextual variables in place. One of the things I wanted to investigate was the adaptability of ROLE in a context (elementary schools), and with a population (elementary school principals, administrators, faculty, and staff), that have not been widely investigated a propos of organizational learning and evaluation and show ROLE’s efficacy as the sole quantitative measure of organizational learning and evaluation in an empirical study. However, one of the great things about the ROLE is that it can be adapted for use across a wide spectrum of organizational types and sectors, but I offer elementary schools as one example of its use.
For those of you who don’t know, the ROLE is a highly reliable (Cronbach’s Alpha = .97) instrument developed by Preskill and Torres in 2000 to operationalize the constructs of organizational learning – culture, leadership, systems and structures, communication, and teams along with an evaluation construct. These constructs constitute the internal elements, or as I call them, contextual variables, that must be in place if an organization is to support and encourage organizational learning and evaluative inquiry. Unlike other quantitative instruments, the ROLE measures organizational learning constructs that are concrete and well-established in the extant literature, reflects the multiple dimensions of organizational learning, and was designed to help you determine an organization’s level of readiness for implementing organizational learning, evaluation practices, and their supporting processes.
This is a brief “snap-shot” of one section of the ROLE as adapted for my study with schools. The item tag was the same as on the original ROLE, but I changed it from “We” to “Faculty & Staff” to reflect school terminology.
Hot Tip: The results from the ROLE can be used to: identify the existence of learning organization characteristics, diagnose interest in conducting evaluation that facilitates organizational learning, identify areas of strength to leverage evaluative inquiry processes, and identify areas in need of organizational change and development.
Rad Resource: If you’re looking for the ROLE, look no farther: Preskill, H., & Torres, R. (2001). The readiness for organizational learning and evaluation instrument (ROLE). In D. Russ-Eft & H. Preskill (Eds.), Evaluation in organizations: A systematic approach to enhancing learning, performance, and change. New York: Basic Books.
The American Evaluation Association is celebrating Organizational Learning & Evaluation Capacity Building (OL-ECB) TIG Week with our colleagues in the OL-ECB AEA Topical Interest Group. The contributions all this week to aea365 come from our OL-ECB TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.






