Hello. I am Karen Widmer, a 4^th year doctoral student in the Evaluation program at Claremont Graduate University. I’ve been developing and evaluating systems for performance (business, education, healthcare, and nonprofits) for a long time. I think organizations are a lot like organisms. While each organization is unique, certain conditions help them all grow. I get enthusiastic about designing evaluations that optimize those conditions!

Theme: My master’s research project looked at evaluation-related activities shared by high-performing organizations. For these organizations, evaluation was tied to decision making. Evaluation activity pulled together knowledge about organizational impact, direction, processes, and developments, and this fed the decisions. The challenge for evaluation is to pool the streams of organizational knowledge most relevant for each decision.

Hot Tip:

• Evaluative thinking identifies the flow of organizational knowledge and this provides decision makers with a point of reference for quality decisions.
• In technical language, Knowledge Flow may mediate or moderate the relationship between evaluative thinking and decision quality. Moreover, the quality of the decision could be measured by the performance outcomes resulting from the decision!

Cool Trick:

• Design your evaluation to follow the flow of knowledge throughout the evaluand lifecycle.
• Document what was learned when tacit knowledge was elicited; when knowledge was discovered, captured, shared, or applied; and knowledge regarding the status quo was challenged. (To explore further, look to the work of: M. Polanyi, I. Becerra-Fernandez, and C. Argyris and D. Schon.)
• For the organizations I looked at, these knowledge activities contained the evaluative feedback desired by decision makers. The knowledge generated at these points told what’s going on.
• For example, tacit perceptions could be drawn out through peer mentoring or a survey; knowledge captured on a flipchart or by software; or a team might “discover” knowledge new to the group or challenge knowledge previously undisputed.

Conclusion: By design or still shot, evaluative thinking can view the flow of knowledge critical to decisions about outcomes. Knowledge Flow offers a framework for connecting evaluation with the insights decision makers want for reflection and adaptive response. Let’s talk about it!

Rad Resource: The Criteria for Performance Excellence is a great government publication that links evaluative thinking so closely with decisions about outcomes that you can’t pry them apart.

Rad resource: Neat quote by Nielsen, Lemire, and Skov in the American Journal of Evaluation (2011) defines evaluation capacity as  “…an organization’s ability to bring about, align, and sustain its objectives, structure, processes, culture, human capital, and technology to produce evaluative knowledge [emphasis added] that informs on-going practices and decision-making to improve organizational effectiveness.”

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Capacity Building · evaluative thinking · knowledge flow · organizational learning

Hi, we are Christine Johnson and Terri Anderson, members of the Massachusetts Patient Centered Medical Home Initiative (MA-PCMHI). MA-PCMHI is Massachusetts’ state-wide, multi-site PCMH demonstration project engaging 46 primary care practices in organizational transformation to adopt the PCMH primary care model.  Our roles as Transformation and Quality Improvement Director (Christine) and Qualitative Evaluation Study Team Lead (Terri) require us to understand the 46 practices’ progress towards PCMH model adoption in distinct yet complementary ways.  Our colleagues sometimes assume that we must remain distant to conduct our best possible work.  Their concerns are that our close working relationship will somehow contaminate the initiative or weaken the evaluation’s credibility.  However, we find that maintaining our connection is vital for success of both of the initiative and the evaluation.  We’d like to share the following:

Lessons Learned:

• Transformation and Quality Improvement (Transformation/QI) and evaluation both seek to understand how the practices best adopt the PCMH model and to describe the practices’ progress.  To promote our mutual interest, we regularly attend each other’s team meetings. Doing so increases the opportunity to share our perspectives on the MA-PCMHI. To date the evaluators have advised some formative project adjustments while the MA-PCMHI intervention team has increased the evaluators’ understanding of the survey and performance data submitted from the practices. Currently, the project team and the evaluators collectively are establishing criteria to select six practices for in-depth site visits.
• Transformation/QI and evaluation often use the same data sources but in different ways.  Specifically, the practices use patient record data in their Plan-Do-Study-Act (PDSAs) cycles then submit the same data for the evaluation’s clinical impact measures.  The practices initially resisted this dual data use.  However, through our Transformation/QI-Evaluator connection we increased the practices’ understanding of how their use of data in the PDSAs improved their clinical performance which in turn improved the evaluation’s ability to report a clinical quality impact. Presently, performance data reporting for clinical impact measures and practices’ use of PDSAs have increased.

Hot Tip: Develop a handout describing the similarities and differences between research, evaluation and quality improvement.  Having this information readily available has helped us to address concerns about bias in the evaluation.

Rad Resources:

• To learn more about Transformation/QI see the Agency for Healthcare Research and Quality’s 2010 Consensus Report on Practice Facilitation for Primary Care improvement.
• To learn more about PDSAs see the Institute for Healthcare Improvement’s Plan-Do-Study-Act tools.

(Share Clip)

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Capacity Building · qualitative

Hello! We are Johanna Morariu, Kat Athanasiades, and Ann Emery from Innovation Network. For 20 years, Innovation Network has helped nonprofits and foundations evaluate and learn from their work.

In 2010, Innovation Network set out to answer a question that was previously unaddressed in the evaluation field—what is the state of nonprofit evaluation practice and capacity?—and initiated the first iteration of the State of Evaluation project. In 2012 we launched the second installment of the State of Evaluation project. A total of 546 representatives of 501(c)3 nonprofit organizations nationwide responded to our 2012 survey.

Lessons Learned–So what’s the state of evaluation among nonprofits? Here are the top ten highlights from our research:

1. 90% of nonprofits evaluated some part of their work in the past year. However, only 28% of nonprofits exhibit what we feel are promising capacities and behaviors to meaningfully engage in evaluation.

2. The use of qualitative practices (e.g. case studies, focus groups, and interviews—used by fewer than 50% of organizations) has increased, though quantitative practices (e.g. compiling statistics, feedback forms, and internal tracking forms—used by more than 50% of organizations) still reign supreme.

3. 18% of nonprofits had a full-time employee dedicated to evaluation.

4. Organizations were positive about working with external evaluators: 69% rated the experience as excellent or good.

5. 100% of organizations that engaged in evaluation used their findings.

6. Large and small organizations faced different barriers to evaluation: 28% of large organizations named “funders asking you to report on the wrong data” as a barrier, compared to 12% overall.

7. 82% of nonprofits believe that discussing evaluation results with funders is useful.

8. 10% of nonprofits felt that you don’t need evaluation to know that your organization’s approach is working.

9. Evaluation is a low priority among nonprofits: it was ranked second to last in a list of 10 priorities, only coming ahead of research.

10. Among both funders and nonprofits, the primary audience of evaluation results is internal: for nonprofits, it is the CEO/ED/management, and for funders, it is the Board of Directors.

Rad Resource—The State of Evaluation 2010 and 2012 reports are available online at for your reading pleasure.

Rad Resource—What are evaluators saying about the State of Evaluation 2012 data? Look no further! You can see examples here by Matt Forti and Tom Kelly.

Rad Resource—Measuring evaluation in the social sector: Check out the Center for Effective Philanthropy’s 2012 Room for Improvement and New Philanthropy Capital’s 2012 Making an Impact.

Hot Tip—Want to discuss the State of Evaluation? Leave a comment below, or tweet us (@InnoNet_Eval) using #SOE2012!

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Capacity Building · Data · Evaluation · nonprofit · organizational learning · reports

Hi!  I’m Sarah Baughman, Evaluation and Research Leader for eXtension.  eXtension is the virtual presence of the Cooperative Extension System. We work with faculty from land grant universities to provide unbiased, research-based educational resources across a wide range of topics including feral hogs, fire ants, and families through our 70 communities of practice. A major aspect of my daily work is assisting communities of practice with evaluation efforts.  Although most faculty are familiar with evaluation basics the virtual work environment tends to confound faculty.

Hot Tip - Back to Basics – When working with faculty on evaluations for programs that involve social media and/or web based resources I take them back to the basics.  I help them situate their social media and virtual “tools” into the context of their programs by asking lots of questions that point back to evaluation basics such as programmatic mission, purpose and goals.  Why are they tweeting?  What do they hope to achieve through by integrating social media into their programs?

Lesson Learned – Capacity building is an on-going process.  The landscape of our work changes rapidly with new faculty on board, new technologies developed and new communities of practice forming.  As one faculty member embraces evaluation as a critical component of their work, another community of practice changes leadership necessitating renewed capacity building efforts.

Lesson Learned – Another key for working with faculty immersed in their disciplines is to show them how evaluation methodologies are similar to their research methods.  The purpose of evaluation is different than research but the methodologies are the same.

Rad Resource – Google + Hangouts have proven to be an invaluable resources for one on one or group meetings.  Hangouts are free video conferences that allow screen sharing and are mobile device friendly so busy faculty can meet from almost anywhere.  The screen sharing allows me to walk through tools with them or troubleshoot issues that are difficult to describe in other contexts.

Rad Resource – There is a lot of information on social media marketing and measurement but it is primarily aimed at for-profit businesses.  In the world of education and non-profits the goals and outcomes can be fuzzy.  Measuring the Networked Nonprofit by Beth Kanter and Katie Delahaye Paine does an excellent job of describing the importance of measuring social media and more importantly, how measurement can help change practice.

Rad Resource – One of the eXtension communities of practice is devoted to helping Extension professionals evaluate programs and demonstrate programmatic impacts. Their work is helpful to non-formal or informal educators, non-profits and anyone working on evaluation in a complex, decentralized environment. Connect with us at @evalcop or blogs.extension.org/evalcop/.

(Share Clip)

The American Evaluation Association is celebrating Extension Education Evaluation (EEE) TIG Week with our colleagues in the EEE Topical Interest Group. The contributions all this week to aea365 come from our EEE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Capacity Building · Extension

Hi! I’m Mary Arnold, a program evaluator with the Extension 4-H Youth Development program at Oregon State University. This AEA 365 entry kicks off a week of postings by members of the Extension Education Evaluation TIG, which is starting its 32^nd year! Extension is the outreach arm of the public Land-Grant universities and its mission is to provide research-based education programming to the public. In essence, we take the university to the people.

The evaluation needs of Extension are vast and complex. From evaluating small local programs, to large grant-funded endeavors, to calculating the public benefit of our programs, as an organization we are always learning about effective evaluation. Extension recently shared its organizational learning through a New Directions for Evaluation Issue devoted to evaluation in complex organizations, which could be useful to other such organizations.

Rad Resource: New Dimensions for Evaluation: Program Evaluation in a Complex Organizational System: Lessons Learned from Cooperative Extension, Volume 210 2008.

In my Extension role, I spend a great deal of time engaged in evaluation capacity building. Over the years I have discovered several important elements of capacity building that lead to success:

Hot Tip: Use a four-fold framework for building evaluation capacity:

1. Develop and use logic models to ensure sound program planning and create evaluation plans. This helps everyone to be clear on the program, its intent, and outcomes.
2. Provide one-on-one help to educators with their own evaluation projects. Learning is best accomplished when applied to a real project that means something to the learner
3. Facilitate small-group collaborations on a real project, allowing members to learn and practice new skills within the cycle of evaluation.
4. Conduct larger scale, multi-site evaluations for your organization that allows everyone to participate at some level.

Rad Resource: My experience using this evaluation capacity framework is detailed in an article in the June 2006 American Journal of Evaluation.

Rad Resource: Building Evaluation Capacity: 72 Activities for Teaching and Training by Hallie Preskill and Darlene Russ-Eft (2005 Sage Publications). This detailed book is my “go-to” whenever I plan a new evaluation capacity building training. The activities are detailed, creative, and engaging for adult learners of evaluation.

Rad Resource: The Logic Model Guide Book: Better Strategies for Great Results (2^nd ed) by Lisa Wyatt Knowlton and Cynthia C. Phillips (2013 Sage Publications). This book focuses on the application of logic models in the real world. Using more complex models in capacity building trainings has helped learners to appreciate more deeply the value of logic modeling as something more than a simple exercise without real meaning.

The American Evaluation Association is celebrating Extension Education Evaluation (EEE) TIG Week with our colleagues in the EEE Topical Interest Group. The contributions all this week to aea365 come from our EEE TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Capacity Building · Extension · training

I’m Cheryl Poth and I am an assistant professor at the Centre for Applied Studies in Measurement and Evaluation in the department of Educational Psychology, Faculty of Education at the University of Alberta in Edmonton, Canada. My area of research is focused on how developmental evaluators build evaluation capacity within their organizations. My use of mixed methods is pragmatically-driven, that is, I use it when the research/evaluation question(s) require the integration of both qualitative themes and quantitative measures to generate a more comprehensive understanding. Most recently, my work within research teams has provided the impetus for research and writing about the role of a mixed methods practitioner within such teams.

Lessons Learned:

• Develop and teach courses. In 2010, I developed (and offered) a doctoral mixed methods research (MMR) course in response to the demand from graduate students for opportunities to gain skills within MMR. The course was oversubscribed and at the end of the term we formed a mixed methods reading group, which continues to provide support as students are working their way through their research process. I am fortunate to be able to offer this course again this winter and already it is full!
• Offer workshops. To help build MMR capacity, I have offered workshops in a variety of locations, most recently at the 9^th Annual Qualitative Research Summer Intensive held in Chapel Hill, NC in late summer and at the 13^th Thinking Qualitatively Workshop Series offered by the International Institute for Qualitative Methodology held in Edmonton, AB in early summer. These workshops remind me of the reality for many researchers that their graduate programs required completion of an advanced research methods course that was either qualitatively- or quantitatively-focused and of the need to build a community of MM researchers and that the community can exist locally or using technology can exist globally! It has been a pleasure to watch as new and experienced researchers begin to learn about MMR designs and integration procedures.
• Join a community. One of the places where I have begun to find my community MM researchers was through a group currently working on forming the International Association of Mixed Methods, at the International Mixed Methods conference, and the mixed methods researchers on Methodspace.

Hot Tip:

• Stay tuned, the International Association of Mixed Methods is in the works and one of their goals is to build MMR capacity by offering learning opportunities using a variety of methods.

Rad Resource:

• Join the AEA Mixed methods TIG (Donna Mertens and Lennise Baptiste are co-chairs)
• Join the mixed methods researcher group at Methodspace
• Link to information about the 2012 Qualitative Research Summer Intensive
• Link to the International Institute for Qualitative Methodology
• Link to the International Mixed Methods conference

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Capacity Building · learning · Professional Development · Workshops

Hello! My name is Laura Keene, owner of Keene Insights, a one-woman consulting shop based in sunny Los Angeles, California. Over the past nine years, I’ve worked as both an internal and an external evaluator in a variety of settings. For many organizations, especially smaller nonprofits, working with me is their first introduction to evaluation. The success of these projects hinges on good capacity building as I work to overcome their fears and give them the tools they need to make design decisions and use findings. (See Reid Zimmerman’s post for a great description of evaluation fears).

Sometimes this work requires sessions that are solely focused on training. Other times we move back and forth between learning, discussion, and decision making. In either case, I try to integrate activities that tap into adult-learning principles and actively engage the group as much as possible. This makes the training more effective and much more fun.

Hot Tips:

In the excitement of planning great activities, don’t forget to keep these basic training principles in mind:

• Put yourself in the shoes of your participants. What is their background? What is their perspective on the topic? What do they know already? What are they expecting to gain from the training overall?
• Keep the purpose of the activity in the forefront. What’s the main point? Why does it matter? All components should support the purpose.
• Be as prepared as humanly possible. Knowing the activity inside and out gives you the flexibility to handle the unexpected and adapt where needed.

Rad Resource:

Building Evaluation Capacity by Hallie Preskill and Darlene Russ-Eft is a great source for activities and ideas. It has step-by-step instructions for activities that focus on everything from exploring the differences between evaluation and research to understanding and interpreting data. Handouts included!

For a more comprehensive training guide check out Bruce Klatt’s Ultimate Training Workshop Handbook.

The American Evaluation Association is celebrating Organizational Learning & Evaluation Capacity Building (OL-ECB) TIG Week with our colleagues in the OL-ECB AEA Topical Interest Group. The contributions all this week to aea365 come from our OL-ECB TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Capacity Building · perspective · training

I’m Joseph Bauer, Chair of the Organizational Learning-Evaluation Capacity Building Topical Interest Group (OL-ECB TIG), here with Allison Titcomb, ALTA Consulting/ Arizona Evaluation Network (AZENET) /Local Affiliate Collaborative (LAC).  We have prepared this post as a model activity that reflects joint purposes of the OL-ECB and the LAC.

The OL-ECB TIG and the Local Affiliate Collaborative, comprised of a Steering Committee and representatives/ liaisons from local and regional evaluation organizations across the country (aka “Affiliates” of AEA), are exploring opportunities to expand and meld our shared interests.  We have similar missions and goals to build and support evaluation capacity among evaluators and we also share an interest in using “Communities of Practice” as a strategy for ongoing learning.

For the OL-ECB, there is the opportunity to leverage community-based experience and information with evaluation capacity-building over a wide area network, as well as enhance knowledge about the dynamic of community-based organizations – to be able to create two-way learning opportunities for problem-solving.

For the Local Affiliates, there is the opportunity to leverage OL-ECB activities and annual Conference sessions to expand the collaborative structure among the Affiliates that are spread across the country.

RAD Resources:

Communities of Practice:

• Ewenger.com/theory

• Cdc.gov/phcommunities

• Ncddr.org/cop/whatiscop.html

E. Wenger, R. McDermott & W.M. Snyder (2002).  Cultivating Communites of Practice: A Guide to Managing Knowledge.  HBR Press.  Particularly, Chapter 3 -7 Principles for Cultivating a Community of Practice.

Community-Based Participatory Research: A Capacity-Building Approach for Policy Advocacy Aimed at Eliminating Health Disparities – Community-based participatory research (CBPR) is a partnership approach that can facilitate capacity building and policy change through equitable engagement of diverse partners.

Using evaluability assessment and evaluation capacity-building to strengthen community-based prevention initiatives - This report illustrates the use of mixed methods to plan, implement and evaluate a model to catalyze Community-Based Organization’s (CBO’s) systematic assessment of prevention initiatives and considerations in evaluation capacity-building.

Get Involved:  We have the idea of enhancing a Community of Practice that will create synergy among our groups – with the philosophy that we are stronger together than separately.  We would love to hear from anyone, particularly those that are members of the OL-ECB or one of the Local Affiliates.  We think there are plenty of good ideas out there – needing people to get involved.  Come share your thoughts.

The American Evaluation Association is celebrating Organizational Learning & Evaluation Capacity Building (OL-ECB) TIG Week with our colleagues in the OL-ECB AEA Topical Interest Group. The contributions all this week to aea365 come from our OL-ECB TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Capacity Building · communities of practice

My name is Alicia McCoy and I am the Research and Evaluation Manager at Family Life in Melbourne, Australia.  Family Life is a community service organization that offers a diverse range of services for families, children and young people, including family support, counseling, and mediation.

In 2009, Family Life established an internal research and evaluation unit. But building a culture that supports evaluation didn’t follow automatically. It’s taken time, patience, focus, and even a little humour.  It’s important to regularly communicate with staff about the value of research and evaluation.  Here are some of my tips and lessons learned for doing this well.

Lesson learned: It helps to have a channel 

The creation of an internal research and evaluation blog in early 2011 turned out to be one of the greatest contributors to the growing research and evaluation culture at Family Life.  Make your blog highly visible and easily accessible to staff.  Update it regularly to keep it fresh and interesting.

Lesson learned:  Think outside the square

Blogs can be used for much more than just posting and comments.  Ours includes current research and policy information, case studies, program evaluation summaries, an acknowledgements register, key resources, relevant links, and even competitions.  It is a one-stop-shop for anything related to research and evaluation.

Hot tip: Be interactive

All staff members are encouraged to interact with the blog by guest-blogging on topics of interest to them and their work, and by commenting on and discussing the contributions of others.  These entries have been some of the most popular and have motivated others to give it a go.  Have a support process in place for those who are less confident with their blogging skills.

Hot tip: Be creative to boost relevance

Our blog delivers research and evaluation to staff in informal and creative ways. Examples include using a media story to provide context for a piece about a current policy initiative, or using an anecdote to provide the ‘hook’ for an entry on a current research study.  Feedback from staff suggests that these kinds of approaches help them better understand research and evaluation and makes it more interesting. The upshot: they are more likely to use information from evaluations and research studies.

Hot tip: Evaluate your own progress

It can take several months or longer for a blog to become part of a culture, especially if your organization is new to blogging.  It is important to regularly measure the popularity and use of the blog to see how you’re tracking.  Successful blogs will gradually increase the number of monthly visits until the majority of staff are engaged.

The American Evaluation Association is celebrating Internal Evaluators TIG Week. The contributions all week come from IE members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluator.

blog · Capacity Building

Hi, I’m Craig Wiles, Senior Consultant at Public Sector Consultants in Lansing, Michigan. I provide research and evaluation services for clients in health and human services, education and the environment. I am sharing a tip on how to use Tableau as part of a data exploration process with a group of stakeholders.

To begin, I did the heavy statistical lifting outside of Tableau, so this would not lapse into a data-mining exercise. In this case, I worked with a state-level stakeholder group to identify data sources, research priorities, and statistically significant correlations in the data. Once we had our short list of correlated variables to explore in more detail, we convened a series of two hour, interactive data exploration sessions. At these sessions, we used Tableau to visually display the data (in this case, educational data), identify high and low performing school districts, and look for other obvious patterns or outliers in the data.

We tended to use stacked bar charts and scatter plots to help with this visual part of the data exploration. One tool in Tableau that was especially helpful in this context was the filter bar. Using the filter tool, we were able to adjust the range of scores we were looking for in our combination of variables according to tolerances set by the stakeholders. For example, we looked for school districts that had a high graduation rate, low dropout rate, and a higher ratio of students with disabilities in general education classrooms.

I recommend using Tableau for data exploration because it is:

• Visual,
• Interactive, and
• Builds capacity and ownership.

I could have presented this data in charts and graphs and led a typical ‘sit-and-get’ meeting and landed at the same place (conceptually) at the end of the day. This kind of visual and interactive process, however, really helped to engage my stakeholders, especially those that are usually averse to numbers and data. Ultimately, this was as much about the process as it was about the data.

After our interactive sessions, this group began a series of local focus group conversations with voluntary school districts to further explore the relationships we identified. This qualitative data has provided depth of detail and rich context to the quantitative relationships we explored together.

Using the filter tool:

Using a scatter plot:

Click to increase size

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Capacity Building · data exploration · software · technology