Greetings. We are Linda Cabral and Laura Sefton from the University of Massachusetts Medical School, Center for Health Policy and Research. We are part of a multi-disciplinary team evaluating the Massachusetts Patient Centered Medical Home Initiative (MA-PCMHI), a state-wide, multi-site demonstration project engaging 46 primary care practices in organizational transformation to adopt the PCMH primary care model.  To adopt a mixed methods approach, this evaluation utilizes 1) multiple surveys targeted at different stakeholders (e.g., staff, patients), 2) analysis of cost and utilization claims, 3) practice site visits, and 4) interviews with Medical Home Facilitators (MHFs).

We wanted to connect data from the TransforMED’s Medical Home Implementation Quotient (MHIQ) survey with our MHF interview data. We did this to better understand the practices’ MA-PCMHI experience. MHFs provide a range of technical assistance to aid their assigned practices in their transformation process, making them a great source of information about their practices’ transformation. In an effort to triangulate our evaluation findings, we presented the MHIQ results to the MHFs as part of a traditional semi-structured interview. Presenting site specific survey data to MHFs served the following purposes:

• It allowed for MHFs to share their reflections on why their practices scored the way they did on various domains;
• It prompted MHFs to point out major differences between their assigned sites;
• Focused the MHFs on providing practice-specific information; and  instead of generalities across all the sites to which they were assigned
• MHFs provided insight into some of the strengths and limitations of the survey instrument.

Lessons Learned

• Sharing survey data and having respondents reflect on it during the course of an interview, connecting data, proved to be a very helpful strategy. Specifically, we received more detailed responses from interviewees by asking “Why do you think Practice ABC scored a 5 on the care coordination module”? vs. “What can you tell me about how Practice ABC is implementing care coordination?” MHFs would make the case for or against why a practice scored the way they did on a particular domain.
• Involving the MHFs as “experts” on their assigned sites increased the MHFs’ investment in the evaluation process and their willingness to participate in future evaluation activities.

Hot Tip

• We held these MHF interviews prior to doing practice site visits. The practice-specific information that MHFs shared with us deepened our familiarity with the sites prior to conducting site visits.

Rad Resources

• Qualitative and Mixed Methods Provide Unique Contributions to Outcomes Research
• National Institutes of Health’s Best Practices for Mixed Methods Research in the Health Sciences

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

mixed methods · qualitative

Hi. We are Kathy Muhr, Aniko Laszlo and Alexis Henry from UMass Medical School’s Center for Health Policy and Research. Through Work Without Limits, a Massachusetts network of employers and providers that aims to increase employment for people with disabilities, we evaluate and promote programs, policies, and practices related to recruitment, retention, accommodation, and career advancement of people with disabilities. One of our efforts has been the development of Regional Employment Collaboratives (RECs), which bring together cross-disability employment service providers to more effectively engage employers and identify ways of producing better employment outcomes.

We conducted a process evaluation of the RECs, asking the question “what does it take to build and sustain a collaborative of disability employment service providers?” using a concept mapping approach. Concept mapping is a participatory, multi-stage, mixed-method approach that, among its many uses, involves stakeholders in describing how programs are developed and implemented. To conduct the evaluation, we invited REC members to participate in “brainstorming” sessions during which they generated statements in response to the above question. Next, members sorted and categorized the statements into similar groups. We then entered the sorted statements into a concept mapping software program, which uses multidimensional scaling and hierarchical cluster analysis to generate a visual representation – a concept map – of how the statements were grouped by the participating members. As a final step, we reviewed our findings with the REC Project Directors, who helped us interpret the data, determine a final number of clusters, and generate meaningful labels that captured the concepts the clusters represented.

The concept map shows the relationship of clusters to each other – clusters closer together on the map represent concepts that are more closely related and those further apart represent concepts less closely related. This approach revealed concepts that were related to the necessary ingredients for building collaboratives in general, as well as concepts that were specific to building collaboratives focused on enhancing employment for people with disabilities. We feel that the concept mapping approach was very effective in getting the first-hand perspectives of the stakeholders involved in building the RECs, and provided us with some strategies for further development and replication of the REC model.

Lesson Learned: Mind the learning curve.Make sure you allow enough time to learn your concept mapping software, and to complete all stages in the concept mapping process.

Lesson Learned: Beware of concept mapping software overload. The Internet provides an extensive list of concept mapping software; some are free and others are not. It is important to select the software that best meets the needs of your project.

Rad Resources: Examples of concept maps and various concept mapping software products.

(Share Clip)

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

concept mapping

My name is Clare Nolan and I work for Harder+Company Community Research, a national consulting firm that specializes in evaluation.  For the past 27 years, we have helped foundations, government agencies, and nonprofits plan and evaluate programs and policies.

Accountable Care Organizations (ACOs) mark a new frontier in how healthcare is delivered in the United States.  ACOs create incentives for health care providers from different organizations to work together to treat individual patients across care settings.  Our firm recently conducted an evaluation of California’s first ACO which serves more than 40,000 members of the California Public Employees’ Retirement System (CalPERS), the nation’s second largest purchaser of healthcare services.

Lesson Learned:  The concept of Triple Aim is a key framework for evaluating the success of health reform.  However, it takes time for ACOs to develop the legal and analytical infrastructure necessary to support analyses of these data.  In the near term, evaluation can play a potentially more valuable role by providing formative feedback on the effectiveness of inter-organizational collaboration among ACO partners.  Our evaluation of the CalPERS ACO identified the following as core competencies for ACO success:

1. Leadership and commitment.  Having executive leaders across partner organizations that are invested in the success of the ACO and demonstrate consistent levels of commitment.

2. Accountability and governance.  Establishing inter-organizational governance systems that enable collaborative decision-making, promote accountability, and support communication.

3. People and teams.  Staffed by individuals who are action-oriented, knowledgeable, and strategic, and managed by empowered leaders with a strong clinical background.

4. Data and information technology.  IT systems that enable seamless data-sharing and information exchange that enables patient-level care coordination.

5. Communication.  Honest, open, and transparent communication that supports learning and problem-solving across organizations.

(Share Clip)

Rad Resource:  The surge of interest in ACOs prompted by health reform has resulted in an explosion of new literature.  We found the following articles helpful because they focus less on ACO development and more on implementation.

• Accountable Care Organizations: A New Model for Sustainable Innovation
• A Framework For Evaluating The Formation, Implementation, And Performance Of ACOs
• Large-System Transformation in Health Care: A Realist Review

Hot Tip:  Evaluators can play a strong role in supporting transformations in the health care system that lead to expanded healthcare access and improved overall health status.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

health reform · healthcare · Public Health

Hello, we are Linda Cabral and Laura Sefton from the Center for Health Policy and Research at UMass Medical School. We often collect qualitative data from interviews and focus groups. One challenge we frequently face is how to quickly and efficiently transcribe audio data. We have experimented using voice recognition software (VRS), and we’d like to share our approach.

You will need headphones, a microphone (stand-alone or attached to a headset), and a computer with audio playback and VRS installed on it. We use Dragon Naturally Speaking Premium Version 11.5 voice recognition software, however other VRS is available. Use of audio playback software will allow you to control the playback speed, so you can slow it down, pause, fast forward, and rewind as needed.

Open the audio file in the playback software and open a new document in the VRS. While listening to the audio via the headphones, repeat what you hear into the microphone. During this step, you can format the document to indicate who is speaking and to add punctuation. Because VRS works best when trained to understand a single voice, a designated team member should repeat all spoken content, regardless of how many voices are in the audio file.

This process will generate a document in the VRS that can be saved to your computer as a Word file. As a final review, read through the Word file while listening to the audio file and make needed corrections. This could be done by another member of the project team as a double check of the document’s accuracy.

Hot Tips:

• Spend time training the VRS to recognize your voice. A few practice sessions with the software may be needed where you can read dummy data into the software in order for it to learn your voice. This will improve the transcription quality, minimizing the time spent editing.
• Train the VRS to recognize project-specific acronyms or terminology prior to starting transcription.

Lessons Learned:

• Often, financial resources for evaluation projects are limited. In an effort to keep the transcription process in-house, our administrative staff transcribed the audio files. By using the VRS and someone from our project team familiar with the data as the designated recorder, we have found savings in time and efficiencies.
• No transcription yet has captured 100% content accurately the first time. Therefore, build in time to listen to the recording and to make manual edits.

Rad Resources:

These resources may be helpful as you explore whether VRS is right for you.

• “The Voice Transcription Technique: Use of Voice Recognition Software to Transcribe Digital Interview Data in Qualitative Research” by Jennifer Matheson

• VRS products Review by consumersearch: “In reviews, it’s generally Dragon vs. Dragon”
• 

  (Share Clip)

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

qualitative · technology · transcription

This is Kim Snyder, Associate at ICF International, Rene Lavinghouze, Evaluation Team Lead for CDC’s Office on Smoking and Health, and Patricia Rieker, Adjunct Professor of Sociology at Boston University and Associate Professor of Psychiatry at Harvard Medical School. We have been investigating public health program infrastructure as an ignored component of the left hand side of logic models.

Evaluators often are asked to focus on outcomes or the right hand side of the logic model. How often is life better for people because of a successful public health program (e.g. fewer heart attacks, less exposure to second hand smoke)? While we value the importance of this type of evaluation, we were concerned that the inputs or foundation of our activities are not fully understood. If we don’t start out with the foundation that enables organizational capacity, how are we supposed to really know what affects the outcomes on the right side of logic models?

Lesson Learned:

• The left hand side of the logic model is something that is rarely defined or explained in public health programs. Take a look at the Office on Smoking and Health’s logic model for eliminating nonsmokers’ exposure to secondhand smoke. Under Inputs, what is meant by “State health department and partners”? If it is interpreted and replicated differently can we expect the same outcomes?

So we decided it was important to define and study what functioning public health program infrastructure (or the foundation of public health outcomes) looks like. Previous work, a literature review across public health programs (see Rad Resource) and data from 19 tobacco control programs were used to further our understanding of functioning program infrastructure.

Building on previous work (that is currently in press with the Journal of Public Health Management & Practice) we define infrastructure as a key component and the foundation or platform that supports capacity, implementation, and sustainability of program initiatives; a definable entity, a cyclical process and part of a larger system that requires constant vigilance to be effectively maintained. Using a grounded theory approach we developed the Component Model of Infrastructure or CMI for short.

Rad Resource: Infrastructure: More Than Platforms For Moving Vehicles available in the American’ Evaluation Association (AEA) Public eLibrary.

Sneak Peek:

We are still refining the CMI and hope to share a final version this year. We define five core components of public health program infrastructure:

• Networked Partnerships,
• Multi-Level Leadership,
• Engaged Data,
• Managed Resources, and
• Responsive Plans/Planning.

We see the CMI as a practical model of public health program infrastructure that could provide the framework that grant planners, evaluators, and program implementers need to measure success, to link infrastructure to capacity, and to increase the likelihood that health achievements will be sustainable.

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

infrastructure · logic models · program development

Hello all! We are Kristi Fuller and Glenn Landers, staff at the Georgia Health Policy Center. The Center is housed within Georgia State University’s nationally ranked Andrew Young School of Policy Studies and provides evidence-based research, program development, and policy guidance.

We gave a roundtable presentation at the American Evaluation Association’s (AEA) 2012 conference focused on ensuring utility in evaluation practice, in which we used the Centers for Medicare & Medicaid’s Money Follows the Person (MFP) demonstration program as an example.  Our current evaluation of the MFP program for the state of Georgia has the potential to last ten years.

Hot Tip: When conducting an evaluation over a long time frame, it is conceivable to get into a pattern and produce reports in which stakeholders begin to lose interest. However, keeping the Joint Committee on Standards for Educational Evaluation (JCSEE) Program Evaluation Standards regarding utility in focus can help evaluators avoid this trap.

Lessons learned:

1. Utility standard 2 emphasizes the importance of devoting adequate attention to all relevant stakeholders. For MFP, regular evaluation steering committee meetings bring diverse perspectives of those interested in results, as well as those impacted by the program. Through this interaction, we gain important information used to plan the evaluation so that it provides benefits to a broad range of stakeholders including program participants, familial advocates, attorneys providing legal assistance, programmatic staff, and nursing facility advocates.
2. Utility standard 5 discusses the importance of providing information relevant to needs that are both known and evolving. Recognizing that as programs develop and grow the needs of the invested parties also change is important for ensuring that what is being studied continues to be of relevance to stakeholders. In our experience with MFP, we’ve found that program personnel are interested in delving into data to understand their clients’ experiences, whereas the state’s Medicaid agency is particularly concerned about how services are being utilized.
3. Utility standard 6 describes utilizing various communication methods to create processes and products that are meaningful for challenging and reinterpreting understandings. Interpretation of data can be done in a myriad of ways, and AEA’s Data Visualization and Reporting TIG provides great ideas. One way that we’ve tried to manage this is through dropping the production of our full report from quarterly to semi-annually, allowing more time to develop dashboards and ad-hoc analyses.

Rad Resource:

Food for Thought:

• What are you doing that works well regarding how you engage stakeholders?
• How are you managing different points of view successfully?
• What do you think works well with your data presentation?
• What could you do either more of or differently?

(Share Clip)

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org . aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

evaluation use · program evaluation standards

Hi, we are Christine Johnson and Terri Anderson, members of the Massachusetts Patient Centered Medical Home Initiative (MA-PCMHI). MA-PCMHI is Massachusetts’ state-wide, multi-site PCMH demonstration project engaging 46 primary care practices in organizational transformation to adopt the PCMH primary care model.  Our roles as Transformation and Quality Improvement Director (Christine) and Qualitative Evaluation Study Team Lead (Terri) require us to understand the 46 practices’ progress towards PCMH model adoption in distinct yet complementary ways.  Our colleagues sometimes assume that we must remain distant to conduct our best possible work.  Their concerns are that our close working relationship will somehow contaminate the initiative or weaken the evaluation’s credibility.  However, we find that maintaining our connection is vital for success of both of the initiative and the evaluation.  We’d like to share the following:

Lessons Learned:

• Transformation and Quality Improvement (Transformation/QI) and evaluation both seek to understand how the practices best adopt the PCMH model and to describe the practices’ progress.  To promote our mutual interest, we regularly attend each other’s team meetings. Doing so increases the opportunity to share our perspectives on the MA-PCMHI. To date the evaluators have advised some formative project adjustments while the MA-PCMHI intervention team has increased the evaluators’ understanding of the survey and performance data submitted from the practices. Currently, the project team and the evaluators collectively are establishing criteria to select six practices for in-depth site visits.
• Transformation/QI and evaluation often use the same data sources but in different ways.  Specifically, the practices use patient record data in their Plan-Do-Study-Act (PDSAs) cycles then submit the same data for the evaluation’s clinical impact measures.  The practices initially resisted this dual data use.  However, through our Transformation/QI-Evaluator connection we increased the practices’ understanding of how their use of data in the PDSAs improved their clinical performance which in turn improved the evaluation’s ability to report a clinical quality impact. Presently, performance data reporting for clinical impact measures and practices’ use of PDSAs have increased.

Hot Tip: Develop a handout describing the similarities and differences between research, evaluation and quality improvement.  Having this information readily available has helped us to address concerns about bias in the evaluation.

Rad Resources:

• To learn more about Transformation/QI see the Agency for Healthcare Research and Quality’s 2010 Consensus Report on Practice Facilitation for Primary Care improvement.
• To learn more about PDSAs see the Institute for Healthcare Improvement’s Plan-Do-Study-Act tools.

(Share Clip)

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Capacity Building · qualitative

Hi – I’m Bonnie Stabile and I am Deputy Editor of the World Medical & Health Policy Journal. As an editor, I’d like to encourage evaluation practitioners and scholars to reach out beyond the evaluation community to share their findings and insights. Particularly in the areas of Health and Health Policy, I believe evaluators have much to offer towards improving the understanding and delivery of health care and related services.

Hot Tips:

• Many content journals welcome articles about evaluation and data-based decision making. For example, we urge members of health related topical interest groups to submit work to World Medical & Health Policy. We would especially welcome contributions from the AEA TIG topical areas:
  • Alcohol, Drug Abuse and Mental Health
  • Community Psychology
  • Disabilities and Other Vulnerable Populations
  • Disaster and Emergency Management Evaluation
  • Health Evaluation
  • Human Services Evaluation
  • Social Work
  • Submit a high quality manuscript. Before submitting, evaluate and rework the material to meet ALL the publication demands of the specific journal.
  • Think outside the box. For most journals, you can submit more than just primary research articles; also welcome are book reviews on works published within the last year or commentaries on recent projects or initiatives.
  • If you receive a revise and resubmit, rise to the occasion. Remember they’re still interested in you! They want to see your manuscript improved and published. Respond to reviewers’ comments positively and constructively. When you send your revised manuscript back to the journal, include a detailed, point-by-point explanation of how you have addressed each of the reviewers’ and editor’s comments.
  • Document primary materials from a variety of sources. Manuscripts with slim documentation raise questions in the minds of reviewers
  • Writing technique matters! Poorly prepared manuscripts suggest sloppy scholarship. Check spelling, syntax, paragraph length (at least three sentences for most), word use, passive voice, and repetition.
  • Format and style. Do your homework. Adhere to the format and style page in the journal or on the journal’s website. For example, the journal may use footnotes rather than endnotes. The same can be said for font, type size, and page organization. Make sure to check if the journal permits headings and subheadings, images, maps, charts, and graphs.
  • Send what the journal requires. Do you need to send paper copies, a thumb drive, or can you submit electronically?Be sure to provide complete contact information, including a postal address, e-mail address, and telephone number. If you move don’t forget to update your contact information.

Rad Resources:

• World Medical & Health Policy, you can find our most recent issue, as well as guidelines for authors
  

• Strunk & White’s The Elements of Style  (link to book on Amazon)

Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

No tags

Hi aea365ers! Susan Staggs here. I’m an Associate Professor of Psychology at the University of Wisconsin – Stout, coordinator of an online graduate certificate program in Evaluation Studies, and incoming Program Director of our Master’s in Applied Psychology program, which includes a concentration in Evaluation Research. I write today on behalf of the American Evaluation Association’s Community Psychology Topical Interest Group (TIG). One of the principles we believe in is evaluation should be an “active collaboration among researchers, practitioners, and community members… undertaken to serve those community members directly concerned, and guided by their needs and preferences, as well as by their active participation.” With that in mind, I’ve chosen to highlight some wonderful community-oriented resources and organizations that have the potential to enrich your evaluations of community health initiatives by enhancing their explicit focus on community collaboration.

Rad Resource – The University of Kansas’ Community Toolbox: This amazing site offers practical, specific guidance for promoting community health and evaluating community health initiatives. There’s a whole slew of How-To Guides on topics such as stakeholder engagement and community assessment. There’s a Troubleshooting area for help solving problems such as dealing with disappointing evaluation results and unintended intervention effects. The Promising Approaches section highlights the latest evidence-based intervention research, while a Connect With Others area lets you ask questions of intervention experts. Specific guidance on Participatory Evaluation is available here, as is an Evaluation Model for Community Initiatives. A treasure trove, loved and respected by community evaluation practitioners.

Clipped from: ctb.ku.edu (share this clip)

Rad Resource – Community-Campus Partnerships for Health: Want to make sure you’re serving the community’s evaluation needs? This community advocacy organization, focused on social justice and equity in partnerships between communities and universities, can help you with that. They host conferences and training opportunities designed to promote equalization of power in collaborative work between community members and academics. A peer mentorship program for community representatives involved in work with academic partners is available, as is guidance on community-centered research ethics. There are many other valuable and practical resources available on the site as well; this is a fabulous site from a wonderful organization whose purpose is strongly rooted in community psychology values. They maintain a very active listserve on ethics in community-based participatory research.

We’re celebrating all this week with our colleagues in the American Evaluation Association Community Psychology Topical Interest Group. The contributions all week come from CP TIG members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org.

No tags

Greetings! My name is Shana Alford and I am an internal program evaluator at ACCESS Community Health Network, a 501 (c) federally qualified health center network of more than 40 health centers in under-served communities within the City of Chicago. As an internal evaluator, I objectively analyze how federally funded community health programs are implemented in targeted communities and determine measurable impact and outcomes.

In my experience, I have learned the art of asking the right questions to promote positive dialogue and build relationships. Let’s face it, sometimes you can be viewed as an outsider, even when you are an insider!  Asking the right questions doesn’t imply there is a set of wrong questions, but questions can stimulate dialogue and serve as a powerful medium for conversation, so it is important to set the stage to gather the information you need. Keep in mind that one or more conversations can lead to positive change or at least increase awareness and learning among staff, management, and you too!

The Art of Asking the Right Evaluation Questions

photo credit: WingedWolf via photopin cc

Hot Tips:Asking questions for evaluation purposes is an art and here are three that I use frequently:

1) Insight: Listen to your program team and learn about their unique experiences, their responsibilities, successes and challenges. When you have insight about a program then you are more likely to probe deeper into areas that seem to raise a red flag, or deserve attention because they are going so well. Staff will find pleasure that you know the ins and outs of their program.

2) Relevance: An effective question will be right for the moment and relevant for the group of people you are addressing. I have learned the hard way that asking questions to management that should be asked to program staff and vice versa can cause an awkward case of silence, frustration, or sometimes misunderstanding. It is important to know your audience.

3) Patience: Asking questions should not feel like an interrogation to staff or management. Also, they may not be able to answer a question or feel uncomfortable for many reasons. Therefore, it is importance to practice patience. If the purpose of the question is to learn something new, highlight an existing issue, or clarify, then the evaluator should give the program team time to respond, even if it is at a later date. Hint: If people are unresponsive to a question, sometimes taking a step back and asking the same question, but differently will yield the results you are looking for. This may sound unlikely, but it is true, try it!

Rad Resources:

• Longwoods™ publishes healthcare research, reports, reviews, commentaries and news from and for academics, scientists, clinicians, policymakers, administrators and pundits.
• The Community Toolbox, University of Kansas

The American Evaluation Association is celebrating the Chicagoland (CEA) Evaluation Association Affiliate Week with our colleagues in the CEA AEA Affiliate. The contributions all this week to aea365 come from our CEA members. Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on the aea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

affiliates · dialogue · Relationships